diff --git a/.agents/learnings.md b/.agents/learnings.md index fbec9d6f5..e07133105 100644 --- a/.agents/learnings.md +++ b/.agents/learnings.md @@ -13,8 +13,9 @@ Reference document for AI agents. These are hard-won insights from past RFC impl - **Duckborrowing is codegen policy**: when work touches lowering/emission, call arguments, collection literals, returns, match scrutinees, Rust interop, or generated `.clone()`s, route ownership through `src/backend/ir/ownership.rs` / `ValueUseSite` and update trait-bound inference/tests instead of adding local `.clone()`, `.as_ref()`, `str(...)`, or `.into()` workarounds. (Issue #121, April 2026) - **Forward receivers by borrow shape**: when lowering wrappers or adapters around methods, model `self` as the callable's actual receiver borrow (`&Owner` or `&mut Owner`) and pass that through directly; inserting `.clone()` hides a compiler lowering shortcut as user-visible ownership behavior and breaks mutable receiver support. (RFC 036 / issue #170) - **PR conflict resolution must use `origin/main` as the merge base**: when a user asks to merge main or resolve PR conflicts, inspect and merge against `origin/main`, not the local `main` branch copy. Local `main` can lag the remote and give a false “merged main” result while GitHub still reports conflicts. (RFC 015 branch, April 2026) +- **Match ladders are a smell**: in authored `.incn` code, avoid nested `match` ladders that only peel `Option`/enum variants before continuing; prefer `if let`, early returns, or a focused `match` with shallow arms. Do not "fix" the ladder by creating a forest of one-use helpers; keep helpers only when they name a real concept. (InQL #25 source-quality cleanup, May 2026) - **Name repeated kind checks**: when the language lacks grouped pattern arms, do not duplicate long `kind == A or kind == B ...` chains across functions; hide the grouping behind one predicate/helper so later enum-surface changes do not drift between call sites. (Prism output-column cleanup, April 2026) -- **Never expose local paths**: Shareable artifacts must use repo-relative paths or plain command names; absolute workstation paths like `/Users/...` leak personal details and should be blocked in hooks and avoided in docs, issues, PR text, and examples. +- **Never expose local paths**: Shareable artifacts must use repo-relative paths or plain command names; absolute workstation paths like `/Users/...` leak personal details and should be blocked in hooks and avoided in docs, issues, PR text, and examples. For GitHub issues/PRs, run the check before the first create/update call because edit history can preserve the original text. - **New AST variants need full pipeline wiring**: adding a `Statement`/`Expr` variant is never parser-only; you must update formatter, feature scanners, typechecker, lowering, and any AST bridge layers in the same change or compilation/tests will break in scattered places (RFC 027 Phase 6). - **Method defaults need emission tests**: method default arguments can pass typechecking but still emit invalid Rust if the method-call emitter does not synthesize omitted defaults; when adding or using method defaults, include a run/codegen test that calls the method with omitted arguments. (Issue #286) - **Stdlib function defaults cross stages**: imported stdlib free-function defaults must be preserved from AST loading through typechecking, lowering, and emission; a typechecker-only default fix can still produce generated Rust calls with missing arguments. Add end-to-end run coverage when public stdlib APIs rely on omitted defaults. (RFC 064 / issue #342) @@ -78,14 +79,16 @@ Reference document for AI agents. These are hard-won insights from past RFC impl - **Implementation docs must be user-facing**: RFCs and release notes do not satisfy user documentation for a new language/compiler feature; when behavior is user-visible, update the authored explanation/how-to/tutorial/reference docs where users actually learn the surface, not just the RFC or changelog. (RFC 049 / issue #333) - **Markdown prose should not be short-wrapped**: when generating authored Markdown documents, do not manually wrap prose to artificial line lengths; use natural paragraph lines unless the structure itself requires line breaks, because short-wrapped prose reads fragmented and creates noisy diffs for whitepapers, RFCs, and research docs. (Pallay research docs, April 2026) - **RFC phase before code**: when using `ralph-loop` for an RFC implementation, move the RFC to `In Progress` and confirm the implementation plan/checklist before writing code; do not treat lifecycle edits and phase confirmation as a post-implementation cleanup step. (RFC 016 / issue #327) +- **RFC PR means implementation PR**: in RFC headers, `RFC PR` is the PR where the RFC was implemented or shipped, not the proposal issue or the PR that first added the Draft RFC document. Leave it unset for Draft or otherwise unimplemented RFCs even when a proposal issue exists. - **North-star first for RFCs**: when a maintainer asks for an RFC, start from the desired end-state contract and only discuss incremental slices after that north-star is explicit; do not reflexively shrink RFC scope into the smallest implementable change unless the user asks for rollout planning. +- **Pre-RFC research lives in root `__research__`**: capture exploratory north-star notes, spikes, and design parking lots under the repository root `__research__/` directory, not `.agents/`; `.agents/` is for reusable agent workflows/learnings rather than project research artifacts. (Android/Incan rustc bridge ideation, May 2026) - **RFCs are decision records, not diaries**: keep RFCs as moment-in-time intent/status documents, and move implementation details, drift notes, and current behavior into regular docs or release notes with issue links instead of rewriting RFC narrative in flight. - **Generated references are gates**: when adding or changing a stdlib namespace or language registry entry, run `cargo run -p incan_core --bin generate_lang_reference` and verify no diff before publishing; `make pre-commit` alone may miss generated `language/reference/language.md` drift that CI enforces. (RFC 065 / issue #343) - **Implementation work must check dev version first**: before landing an implementation on the active dev line, verify the repo's actual source-of-truth version instead of assuming an older release train from stale docs or a worker worktree; at minimum, implementation work should bump `-dev.N` by one and update any versioned docs/release-note targets that track `main`. (Issue #333, April 2026) - **Stdlib closeouts need reference-nav parity**: when a stdlib issue changes a module's implementation shape or canonical docs path, update the stdlib reference index, MkDocs nav, and any legacy standalone reference page together; otherwise modules like `std.testing` drift out of the `language/reference/stdlib/` structure even when release notes and how-to docs were updated. (Issues #301/#302) - **RFC lifecycle edits need graph updates**: When an RFC is renamed, moved, split, or superseded, update inbound RFC references and regenerate `workspaces/docs-site/docs/_snippets/rfcs_refs.md` plus `workspaces/docs-site/docs/_snippets/tables/rfcs_index.md`; otherwise the docs graph silently points at stale RFC paths and statuses. (RFC 012/050/051 split) - **RFC checklist gaps force replanning**: In a Ralph loop, unchecked RFC `Progress Checklist` items are scope failures, not PR-body residual risks; route them back through Plan -> Do -> Check -> Act before publishing, and only use closing keywords after the RFC is fully checked and bumped. (RFC 084 / issue #453) -- **Ralph worktrees live in encero/tmp**: for `ralph-loop`, every implementation must start in a fresh worktree under `/Users/danny/Development/encero/tmp`, not `/tmp` and not the primary checkout, so VS Code discovers the workspace and orchestration stays consistent. (RFC 016 / issue #327) +- **Ralph worktrees live in encero/tmp**: for `ralph-loop`, every implementation must start in a fresh worktree under the workspace root's `encero/tmp` directory, not a system temp directory and not the primary checkout, so VS Code discovers the workspace and orchestration stays consistent. (RFC 016 / issue #327) ## Builtin trait stubs and stdlib method lookup (#193) diff --git a/.agents/skills/create-github-issue/SKILL.md b/.agents/skills/create-github-issue/SKILL.md index 3069f56cb..ec65e1d5f 100644 --- a/.agents/skills/create-github-issue/SKILL.md +++ b/.agents/skills/create-github-issue/SKILL.md @@ -25,7 +25,30 @@ description: Drafts a GitHub issue title and body using the target repository's 6. **Produce the draft** — See [Output format](#output-format). For YAML `body` block semantics (markdown vs textarea vs dropdown vs checkboxes), use [reference.md](reference.md). -7. **Optional: related PR or branch** — If the issue tracks follow-up work, mention the branch or PR link in the body where the template has a freeform section. +7. **Run the public text safety gate** — Before showing the draft to the user or calling any GitHub issue creation/update tool, inspect the exact title and body that will be published. Public issue text must not contain local absolute paths, personal workspace paths, usernames from local paths, machine-specific temporary directories, shell prompts, or environment details that are not needed to reproduce the issue. Replace them with repo-relative paths, generic commands, or neutral placeholders. + +8. **Optional: related PR or branch** — If the issue tracks follow-up work, mention the branch or PR link in the body where the template has a freeform section. + +## Public Text Safety Gate + +GitHub issues are public by default and edits may remain visible in history. Treat the first publication as permanent. + +Before creating or updating an issue, manually scan the title and body for these banned patterns: + +- local absolute paths, including `/Users/...`, `/home/...`, `/private/...`, `/tmp/...`, and `C:\Users\...` +- personal workspace segments copied from a local checkout path +- commands that invoke a binary through an absolute local path +- local machine usernames, hostnames, shell prompts, or editor-specific transient paths +- private notes, agent state paths, scratch files, or temporary repro directories + +Use these replacements instead: + +- repo-relative paths such as `examples/session_read_transform_write_csv.incn` +- generic commands such as `incan run examples/session_read_transform_write_csv.incn` +- neutral environment descriptions such as `macOS`, `Linux`, `release/v0.3`, or `Incan 0.3.0-rc6` +- short repro files embedded directly in the issue body when possible + +If the only known command uses an absolute local path, rewrite it before publication. Do not publish first and clean it up afterward. ## Fallbacks @@ -88,3 +111,4 @@ Actual: Compiler panics with ... - [ ] Dropdown and checkbox options match **that file’s** YAML, not another project’s. - [ ] Required sections are filled or explicitly flagged as missing. - [ ] Title prefix and labels match the YAML when present. +- [ ] Public text safety gate passed on the exact issue title/body before publishing. diff --git a/.agents/skills/flag-compiler-bug/SKILL.md b/.agents/skills/flag-compiler-bug/SKILL.md index e9023ea45..e4eb9a1aa 100644 --- a/.agents/skills/flag-compiler-bug/SKILL.md +++ b/.agents/skills/flag-compiler-bug/SKILL.md @@ -45,7 +45,7 @@ Do not flag a compiler bug when the issue is more likely: Capture: -- exact command +- exact local command for your private working notes, then derive a sanitized public command before filing - exact observed output, panic text, or wrong behavior - affected stage if inferable: parser, typechecker, lowering, emission, runtime boundary, formatter, CLI, or LSP - current branch / commit / task context @@ -100,7 +100,7 @@ Include: - minimal repro - expected vs actual behavior -- exact command +- sanitized command, using repo-relative paths and tool names instead of local absolute binary paths - logs / panic text / snapshot diff if relevant - affected stage - blocker status @@ -108,6 +108,8 @@ Include: - environment and commit context - related issue, RFC, branch, or task +Before creating the issue, run the `create-github-issue` public text safety gate on the exact title/body you will publish. Do not publish absolute local paths such as `/Users/...`, `/home/...`, `/private/...`, `/tmp/...`, or commands that expose a local checkout path. If the private reproduction used a local compiler binary, publish a generic equivalent such as `incan run path/to/repro.incn` and keep commit/version information in the Environment section. + If the current workflow permits creating the GitHub issue directly, do that after the duplicate check. Otherwise return the ready-to-file draft. ### 6. Return to the original task @@ -138,5 +140,6 @@ If a real workaround exists, continue the task and explicitly record: - Repro is minimal and copy-pastable. - Duplicate search is explicit, not assumed. +- Public issue text is sanitized before the first GitHub create/update call. - Blocking vs workaround judgment is stated plainly. - The original task is either paused honestly or resumed with a real workaround. diff --git a/.agents/skills/review-architecture/SKILL.md b/.agents/skills/review-architecture/SKILL.md index 467cc6f0e..898438bfd 100644 --- a/.agents/skills/review-architecture/SKILL.md +++ b/.agents/skills/review-architecture/SKILL.md @@ -24,7 +24,7 @@ Do not own: - test style - final branch-clean judgment -## Output artifact +## Output artifacts Write a slice report at: @@ -32,6 +32,20 @@ Write a slice report at: Do not write to the canonical `.agents/state/review-report.md`. +When the architecture report has findings, also copy the scope and findings into a lightweight central snapshot outside the repo/worktree under: + +- `/tmp/incan-review-findings/` + +Use a deterministic, descriptive filename when possible: + +- `YYYY-MM-DD-pr--architecture-.md` +- `YYYY-MM-DD-branch--architecture.md` +- `YYYY-MM-DD-review-architecture.md` when no PR or branch context is known + +Do not create a snapshot for clean reviews. The snapshot is raw corpus for later analysis, not canonical guidance. +Treat `/tmp/incan-review-findings/` as an append-only central corpus for local review work: create new snapshot files, but do not delete, overwrite, prune, or "clean up" existing snapshots unless the user explicitly asks for that exact maintenance. +Snapshot content is evidence, not a fix log. Preserve the original finding blocks verbatim, including severity, category, file path, line reference, and explanatory text. If the finding is fixed before the snapshot is written, keep the original finding as observed and add a separate `Resolution` note after it; do not replace the evidence with a resolved checklist item or a summary of the fix. + ## Workflow 1. Review touched code by subsystem, not merely by file. @@ -46,6 +60,7 @@ Do not write to the canonical `.agents/state/review-report.md`. - maintainability warnings, - or design tensions. All three are valid findings. Classify them so downstream fixers know how to treat them, but do not suppress them. +6. If the report contains findings, create `/tmp/incan-review-findings/` if needed and write a new snapshot containing only the review source metadata, Scope, and Findings sections. Copy the finding blocks verbatim from `.agents/state/review-report.architecture.md`; do not generalize them into policy or rewrite them as fix summaries. Preserve exact file:line evidence when the report has it; if a finding is only file-level, make that explicit in the finding text. Do not overwrite an existing snapshot path; add a suffix if needed. ## Slice report shape @@ -69,3 +84,26 @@ Do not write to the canonical `.agents/state/review-report.md`. ``` If there are no findings, say so explicitly. + +## Findings snapshot shape + +Only write this file when findings are present. + +```md +# Architecture Findings Snapshot + +- source: PR # / +- date: YYYY-MM-DD +- reviewer: review-architecture + +## Scope +- reviewed subsystems: + - ... + +## Findings +- [ ] warning | design-tension | wrong layer | src/cli/commands/lifecycle.rs:210 + Resolution policy duplicates env semantics that should stay in `src/project_lifecycle/**`. + +## Resolution +- +``` diff --git a/.agents/skills/review-incan-source-quality/SKILL.md b/.agents/skills/review-incan-source-quality/SKILL.md index 470fc640b..1a23b0b9e 100644 --- a/.agents/skills/review-incan-source-quality/SKILL.md +++ b/.agents/skills/review-incan-source-quality/SKILL.md @@ -37,7 +37,7 @@ Do not own: - docs truthfulness outside comments/docstrings embedded in source - final branch-clean judgment -## Output artifact +## Output artifacts Write a slice report at: @@ -45,6 +45,20 @@ Write a slice report at: Do not write to the canonical `.agents/state/review-report.md`. +When the source-quality report has findings, also copy the scope and findings into the shared lightweight central snapshot folder outside the repo/worktree: + +- `/tmp/incan-review-findings/` + +Use a deterministic, descriptive filename when possible: + +- `YYYY-MM-DD-pr--incan-source-quality-.md` +- `YYYY-MM-DD-branch--incan-source-quality.md` +- `YYYY-MM-DD-review-incan-source-quality.md` when no PR or branch context is known + +Do not create a snapshot for clean reviews. The snapshot is raw corpus for later analysis, not canonical guidance. +Treat `/tmp/incan-review-findings/` as an append-only central corpus for local review work: create new snapshot files, but do not delete, overwrite, prune, or "clean up" existing snapshots unless the user explicitly asks for that exact maintenance. +Snapshot content is evidence, not a fix log. Preserve the original finding blocks verbatim, including severity, category, file path, line reference, and explanatory text. If the finding is fixed before the snapshot is written, keep the original finding as observed and add a separate `Resolution` note after it; do not replace the evidence with a resolved checklist item or a summary of the fix. + ## Review standard Treat touched Incan source as user-facing language showcase code, especially under `crates/incan_stdlib/stdlib/`, examples, fixtures that teach behavior, and RFC-backed language features. @@ -109,6 +123,7 @@ Flag Incan source that has: - `@rust.extern`, `rusttype`, or `rust.module` used to avoid writing expressible Incan behavior; - design narrowing or backend fallback justified by “Incan cannot do this” without local examples, tests, or probe evidence; - sentinel initialization such as `value = 0` only to satisfy later branch assignment; +- nested `match` ladders that only peel `Option`/enum variants before continuing, when `if let`, early returns, or a focused shallow `match` would state the same control flow more directly; also flag helper forests that merely hide the ladder one branch at a time; - verbose `match` blocks that just rewrap a `Result` where `?` would read naturally; - verbose `match` blocks that only transform one `Result` branch where RFC 070 combinators such as `map`, `map_err`, `and_then`, or `or_else` would state the intent directly; - unnecessary type noise when inference or a local helper would be clearer; @@ -135,6 +150,7 @@ Flag Incan source that has: 8. Inspect comments/docstrings last as part of source quality, not as a separate docs-only pass. Short or non-descriptive docstrings are findings even when every declaration technically has one. 9. For each finding, explain what a Pythonic/Incan-native version would make clearer. Do not demand style churn when the existing shape is already direct and readable. 10. Stay report-only unless the user explicitly asks for fixes. +11. If the report contains findings, create `/tmp/incan-review-findings/` if needed and write a new snapshot containing only the review source metadata, Scope, and Findings sections. Copy the finding blocks verbatim from `.agents/state/review-report.incan-source-quality.md`; do not generalize them into policy or rewrite them as fix summaries. Preserve exact file:line evidence when the report has it; if a finding is only file-level, make that explicit in the finding text. Do not overwrite an existing snapshot path; add a suffix if needed. ## Slice report shape @@ -168,3 +184,26 @@ Finding severities: - `note`: cleanup is optional but useful if the file is already being edited. If there are no findings, say so explicitly. + +## Findings snapshot shape + +Only write this file when findings are present. + +```md +# Incan Source Quality Findings Snapshot + +- source: PR # / +- date: YYYY-MM-DD +- reviewer: review-incan-source-quality + +## Scope +- assigned files: + - crates/incan_stdlib/stdlib/uuid.incn + +## Findings +- [ ] warning | source-quality | Rust-shaped sentinel read | crates/incan_stdlib/stdlib/uuid.incn:117 + The function initializes a placeholder byte and overwrites it from a match arm. A direct helper returning `Result[u8, UuidError]` would read like authored Incan rather than generated Rust-shaped control flow. + +## Resolution +- +``` diff --git a/.config/nextest.toml b/.config/nextest.toml index 5f040bc2c..9101ea19e 100644 --- a/.config/nextest.toml +++ b/.config/nextest.toml @@ -4,6 +4,9 @@ [store] dir = "target/nextest" +[test-groups] +nested-cargo = { max-threads = 12 } + [profile.default] # Fail fast: stop after the first test failure during local development. fail-fast = true @@ -20,3 +23,11 @@ status-level = "slow" final-status-level = "slow" slow-timeout = "30s" leak-timeout = "2s" + +[[profile.default.overrides]] +filter = 'binary_id(incan::integration_tests) | binary_id(incan::cli_integration) | binary_id(incan::std_encoding_algorithm_modules) | binary_id(incan::generated_rust_artifact_tests) | binary_id(incan::generated_rust_callability_artifact_tests) | binary_id(incan::generated_rust_native_consumer_tests)' +test-group = 'nested-cargo' + +[[profile.ci.overrides]] +filter = 'binary_id(incan::integration_tests) | binary_id(incan::cli_integration) | binary_id(incan::std_encoding_algorithm_modules) | binary_id(incan::generated_rust_artifact_tests) | binary_id(incan::generated_rust_callability_artifact_tests) | binary_id(incan::generated_rust_native_consumer_tests)' +test-group = 'nested-cargo' diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e5009a7e4..3e8da6aeb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,9 +2,9 @@ name: CI on: push: - branches: [ main ] + branches: [ main, release/** ] pull_request: - branches: [ main ] + branches: [ main, release/** ] env: CARGO_TERM_COLOR: always diff --git a/Cargo.lock b/Cargo.lock index 7a138f40a..44e46622a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -92,7 +92,7 @@ version = "1.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -103,7 +103,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" dependencies = [ "anstyle", "once_cell_polyfill", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -610,27 +610,27 @@ dependencies = [ [[package]] name = "cranelift-assembler-x64" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f8628cc4ba7f88a9205a7ee42327697abc61195a1e3d92cfae172d6a946e722e" +checksum = "008f1a8d1da5074ad858f398775a6d1989031892e46927df5ed18d3be1ed8717" dependencies = [ "cranelift-assembler-x64-meta", ] [[package]] name = "cranelift-assembler-x64-meta" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d582754487e6c9a065a91c42ccf1bdd8d5977af33468dac5ae9bec0ce88acb3e" +checksum = "9fd76237df1f4e26edb5ad7971d20280ed1e193331fd257f1b4e4dfefd88dda2" dependencies = [ "cranelift-srcgen", ] [[package]] name = "cranelift-bforest" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fb59c81ace12ee7c33074db7903d4d75d1f40b28cd3e8e6f491de57b29129eb9" +checksum = "380f0bc43e535df6855bbee649efb00bde39c3f33434c47c8e10ac836d21bf47" dependencies = [ "cranelift-entity", "wasmtime-internal-core", @@ -638,9 +638,9 @@ dependencies = [ [[package]] name = "cranelift-bitset" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f25c06993a681be9cf3140798a3d4ac5bec955e7444416a2fdc87fda8567285d" +checksum = "4811e3e4502de04257e90c0a93225b56d9b85e0f9ad10b81446b415511009610" dependencies = [ "serde", "serde_derive", @@ -649,9 +649,9 @@ dependencies = [ [[package]] name = "cranelift-codegen" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "27b61f95c5a211918f5d336254a61a488b36a5818de47a868e8c4658dce9cccc" +checksum = "82ffadb34d497f3e76fb3b4baf764c24ba8a51512976a1b77f78bdbf8f4aa687" dependencies = [ "bumpalo", "cranelift-assembler-x64", @@ -677,9 +677,9 @@ dependencies = [ [[package]] name = "cranelift-codegen-meta" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0b85aa822fce72080d041d7c2cf7c3f5c6ecdea7afae68379ba4ef85269c4fa5" +checksum = "be4f6992eb6faf086ddc7deaaa5f279abfe7f5fd5ae5709bd38253450fc7b945" dependencies = [ "cranelift-assembler-x64-meta", "cranelift-codegen-shared", @@ -690,24 +690,24 @@ dependencies = [ [[package]] name = "cranelift-codegen-shared" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "833eb9fc89326cd072cc19e96892f09b5692c0dfe17cd4da2858ba30c2cd85c0" +checksum = "70e1b2aad7d055925a4ea9cdbfa9d1d987f9dfc8ad6b708be28f901ac620a298" [[package]] name = "cranelift-control" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9d005320f487e6e8a3edcc7f2fd4f43fcc9946d1013bf206ea649789ac1617fc" +checksum = "89a355348325e0a63b65c00def3871597b9fcc79d25456397010d16d872b3772" dependencies = [ "arbitrary", ] [[package]] name = "cranelift-entity" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5e62ef34c6e720f347a79ece043e8584e242d168911da640bac654a33a6aaaf5" +checksum = "43f4847d93ce2c80d2bff929aa1004dfb3ce2cf5d881f6ced54b8d654d967ba3" dependencies = [ "cranelift-bitset", "serde", @@ -717,9 +717,9 @@ dependencies = [ [[package]] name = "cranelift-frontend" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dfa2ad00399dd47e7e7e33cb1dc23b0e39ed9dcd01e8f026fc37af91655031b8" +checksum = "ba24e5fe5242cc445e7892ef0a51a4351cf716e3a04ac7a3a05820d056c39818" dependencies = [ "cranelift-codegen", "log", @@ -729,15 +729,15 @@ dependencies = [ [[package]] name = "cranelift-isle" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "02c51975ed217b4e8e5a7fd11e9ec83a96104bdff311dddcb505d1d8a9fd7fc6" +checksum = "89bc2035de85c4f04ba7bd57eb5bd3a8b775235bf28852dbf87105115cb8919a" [[package]] name = "cranelift-native" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f9b1889e00da9729d8f8525f3c12998ded86ea709058ff844ebe00b97548de0e" +checksum = "5ea6630c16921ab087792750f239d0c0173411e80179ca7c0ce0710ce9e7646a" dependencies = [ "cranelift-codegen", "libc", @@ -746,9 +746,9 @@ dependencies = [ [[package]] name = "cranelift-srcgen" -version = "0.131.1" +version = "0.131.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d5a8f82fd5124f009f72167e60139245cd3b56cfd4b53050f22110c48c5f4da1" +checksum = "faa4bbad54fc28cc0da1f9a5d7f7f826ec8cafda3d503b401b2daaaa93c63ef0" [[package]] name = "crc32fast" @@ -948,7 +948,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -1478,7 +1478,7 @@ dependencies = [ [[package]] name = "incan" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "blake2", "blake3", @@ -1527,14 +1527,14 @@ dependencies = [ [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "proc-macro2", "quote", @@ -1543,14 +1543,14 @@ dependencies = [ [[package]] name = "incan_semantics_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "incan_core", ] [[package]] name = "incan_semantics_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "incan_core", "incan_semantics_core", @@ -1558,7 +1558,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "axum", "incan_core", @@ -1572,7 +1572,7 @@ dependencies = [ [[package]] name = "incan_syntax" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "incan_core", "incan_semantics_core", @@ -1593,7 +1593,7 @@ dependencies = [ [[package]] name = "incan_web_macros" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "proc-macro2", "quote", @@ -2005,7 +2005,7 @@ version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "536bfad37a309d62069485248eeaba1e8d9853aaf951caaeaed0585a95346f08" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -2047,7 +2047,7 @@ version = "0.50.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -2298,9 +2298,9 @@ dependencies = [ [[package]] name = "pulley-interpreter" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b9326e3a0093d170582cf64ed9e4cf253b8aac155ec4a294ff62330450bbf094" +checksum = "dff0ead8b4616f81b3d3efd41ce41bcf9ea364a5d8df8be8a8a1f98b50104349" dependencies = [ "cranelift-bitset", "log", @@ -2310,9 +2310,9 @@ dependencies = [ [[package]] name = "pulley-macros" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "00c6433917e3789605b1f4cd2a589f637ff17212344e7fa5ba99544625ba52c7" +checksum = "f4389e5820b1b39810ac12a27aa665320cab3caa51913a79637c06f284cfe223" dependencies = [ "proc-macro2", "quote", @@ -3151,7 +3151,7 @@ dependencies = [ [[package]] name = "rust_inspect" -version = "0.3.0-dev.51" +version = "0.3.0-rc20" dependencies = [ "hex", "incan_core", @@ -3239,7 +3239,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys 0.12.1", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -3537,7 +3537,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -3645,7 +3645,7 @@ dependencies = [ "getrandom 0.4.2", "once_cell", "rustix 1.1.4", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -4312,9 +4312,9 @@ dependencies = [ [[package]] name = "wasmtime" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "372db8bbad8ec962038101f75ab2c3ffcd18797d7d3ae877a58ab9873cd0c4bd" +checksum = "af4eccc0728f061979efa8ff4c962cff7041fead4baadb74973f01b9c47158a4" dependencies = [ "addr2line 0.26.1", "async-trait", @@ -4354,9 +4354,9 @@ dependencies = [ [[package]] name = "wasmtime-environ" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e15aa0d1545e48d9b25ca604e9e27b4cd6d5886d30ac5787b57b3a2daf85b57" +checksum = "7e84dbe3208c1336a41546beb75927b3b37e2e4fce06653d214b407136fbe295" dependencies = [ "anyhow", "cpp_demangle", @@ -4385,9 +4385,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-component-macro" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c136cb0d2d47850d6d04a58157130ac98b0df4c17626cd30b083d26b607b7027" +checksum = "c223bd503db76df8d74d1fcca39e734d25f7a0c1dcaf1509b67f3855d1b0f803" dependencies = [ "anyhow", "proc-macro2", @@ -4400,15 +4400,15 @@ dependencies = [ [[package]] name = "wasmtime-internal-component-util" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "49df3d3b4fa2119c6fd161e475b4e21aaefb51d082353b922b433bea37facc65" +checksum = "ab123ad511483a1b918399789d0cc7dea7c5c6476743df73949007b5b225fc74" [[package]] name = "wasmtime-internal-core" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8f2c7fa6523647262bfb4095dbdf4087accefe525813e783f81a0c682f418ce4" +checksum = "4364d345719bba7fc4c435992ea1cb0c118f1e90a88c6e6f22a7a4fc507700c6" dependencies = [ "hashbrown 0.16.1", "libm", @@ -4417,9 +4417,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-cranelift" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "98c032f422e39061dfc43f32190c0a3526b04161ec4867f362958f3fe9d1fe29" +checksum = "c5a3bc28a172037c7864128bb208017a02bba659a59c27acacc048c09e25c1fc" dependencies = [ "cfg-if", "cranelift-codegen", @@ -4444,9 +4444,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-fiber" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d8dd76d80adf450cc260ba58f23c28030401930b19149695b1d121f7d621e791" +checksum = "3c90a899a47d3da6e384e7b4cad61fdcb27535a395742b32440bdf9980ea83fa" dependencies = [ "cc", "cfg-if", @@ -4459,9 +4459,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-jit-debug" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ab453cc600b28ee5d3f9495aa6d4cb2c81eda40903e9287296b548fba8b2391d" +checksum = "84f364747aa74c686b18925918e5cfd615a73c9613c7a31fc1cd86f42df12fbe" dependencies = [ "cc", "wasmtime-internal-versioned-export-macros", @@ -4469,9 +4469,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-jit-icache-coherence" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6a1859e920871515d324fb9757c3e448d6ed1512ca6ccdff14b6e016505d6ada" +checksum = "c3ba98c1492f530833e0d3cc17dbb0c3c57c9f1bb3b078ae44bb55a233e43eba" dependencies = [ "cfg-if", "libc", @@ -4481,9 +4481,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-unwinder" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f1dfe405bd6adb1386d935a30f16a236bd4ef0d3c383e7cbbab98d063c9d9b73" +checksum = "94b8f8a89e8f3660646f820c7d8310a67094156bb866e9d56f1b00892e011206" dependencies = [ "cfg-if", "cranelift-codegen", @@ -4494,9 +4494,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-versioned-export-macros" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2a9b9165fc45d42c81edfe3e9cb458e58720594ad5db6553c4079ea041a4a581" +checksum = "7a12754f1ffc4a3300d56d324c418b8b32cf029606618da22c7d076213882a3f" dependencies = [ "proc-macro2", "quote", @@ -4505,9 +4505,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-winch" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "95f439b70ba3855a8c808d2cd798eef79bcd389f78aa48a8a694ea8e2904410c" +checksum = "4b06e4ed07adc579645e5c55c67b3138c49da2e468fad52d3db7b7a098ecc733" dependencies = [ "cranelift-codegen", "gimli 0.33.0", @@ -4522,9 +4522,9 @@ dependencies = [ [[package]] name = "wasmtime-internal-wit-bindgen" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "17c7ced16dc16d2027f9f8d3a503e191dcce0f53fe9218e7990135b31f8f6fdb" +checksum = "0f08787948e3c983799d616ef7dd57463253e9ca8bab6607eef8134f12353f70" dependencies = [ "anyhow", "bitflags 2.11.0", @@ -4535,9 +4535,9 @@ dependencies = [ [[package]] name = "wasmtime-wasi" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d3d57dd833d0c3ea2016a2aa54c6c517bf8dad9e79d8a593b0252c12bc961e3" +checksum = "1b2f19834bc6edbc31ac95fdcfd5ddcd7643759265a1d545dec36ac6cc788ca8" dependencies = [ "async-trait", "bitflags 2.11.0", @@ -4565,9 +4565,9 @@ dependencies = [ [[package]] name = "wasmtime-wasi-io" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6650bb4c61012b2221e751b7bc1162c7fd11bd1bc29e0714ad6ca463777a3422" +checksum = "c3e0c6efdbaf90906016be9ed9ff17b7b58f393876287beebe5bd7fa1de54dbb" dependencies = [ "async-trait", "bytes", @@ -4609,9 +4609,9 @@ dependencies = [ [[package]] name = "wiggle" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f878b066ad36054ad6e7724230f28ea7f981f44e595e39946d5225fd9e87755" +checksum = "17b644ab90da80bbca28973192978ac452cbd876955bb209e6ff2cd1955e43a7" dependencies = [ "bitflags 2.11.0", "thiserror 2.0.18", @@ -4623,9 +4623,9 @@ dependencies = [ [[package]] name = "wiggle-generate" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f57f0bc709dacc9c69869006457ab4e1bc9d93695400f06224f33cbe8af81778" +checksum = "521f9d558365357274d960340eb9eb4f4d768fafdc79f381fd2e13a85b925ebc" dependencies = [ "heck", "proc-macro2", @@ -4637,9 +4637,9 @@ dependencies = [ [[package]] name = "wiggle-macro" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "63976fe41647f7c55c680b88a7b9b68aae9184f5a6b4a0971bf3eb39c287467f" +checksum = "8a386e86021363c9f0abd1e189e8f8a729d9b5aab2bb7172a3e40f2ab647a936" dependencies = [ "proc-macro2", "quote", @@ -4653,14 +4653,14 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] name = "winch-codegen" -version = "44.0.1" +version = "44.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6da7c536f3cfe5ff63537f795902fed56b8b5adcc7a87843a86dd8d4e57a7946" +checksum = "f16496e92d2b232f9d195ae74f71a674aabae7b7fa722d39068836723d3b653c" dependencies = [ "cranelift-assembler-x64", "cranelift-codegen", diff --git a/Cargo.toml b/Cargo.toml index 775d7f1cd..c3b452e45 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -22,7 +22,7 @@ resolver = "2" ra_ap_proc_macro_api = { path = "crates/third_party/ra_ap_proc_macro_api" } [workspace.package] -version = "0.3.0-dev.51" +version = "0.3.0-rc20" description = "The Incan programming language compiler" edition = "2024" rust-version = "1.92" diff --git a/Makefile b/Makefile index d1943a25a..ce81dc66c 100644 --- a/Makefile +++ b/Makefile @@ -6,12 +6,16 @@ TEST_VERBOSE ?= 0 ifeq ($(strip $(NEXTEST)),) ifeq ($(TEST_VERBOSE),1) -TEST_CMD = cargo test --all --verbose +TEST_CMD = cargo test --all --features lsp --verbose else -TEST_CMD = cargo test --all +TEST_CMD = cargo test --all --features lsp endif else -TEST_CMD = cargo nextest run --all --status-level all +ifeq ($(TEST_VERBOSE),1) +TEST_CMD = cargo nextest run --all --features lsp --status-level all +else +TEST_CMD = cargo nextest run --all --features lsp --status-level slow --final-status-level slow +endif endif # After `make build` / `make build-fast`, symlink ~/.cargo/bin/incan → target/debug/incan so `incan` on PATH (IDE run, @@ -202,7 +206,6 @@ pre-commit-full-gate: t2=$$(date +%s); \ echo "\033[1mRunning tests...\033[0m"; \ $(TEST_CMD); \ - cargo test --features lsp unchecked_lookup_hover --lib; \ echo "\033[32mDONE\033[0m"; \ t3=$$(date +%s); \ echo "\033[1mRunning clippy...\033[0m"; \ @@ -322,7 +325,6 @@ smoke-test-benchmarks-incan: .PHONY: smoke-test-core smoke-test-core: @$(MAKE) smoke-test-release - @$(MAKE) test-rust-inspect @$(MAKE) smoke-test-canary @$(MAKE) smoke-test-web-example @$(MAKE) smoke-test-nested-project-example diff --git a/crates/incan_core/src/bin/generate_lang_reference.rs b/crates/incan_core/src/bin/generate_lang_reference.rs index 4e6e451e3..1935a304c 100644 --- a/crates/incan_core/src/bin/generate_lang_reference.rs +++ b/crates/incan_core/src/bin/generate_lang_reference.rs @@ -481,7 +481,7 @@ fn render_decorators_section(out: &mut String) { start_section(out, "## Decorators"); out.push_str( - r#"User-defined decorators are valid on top-level `def` / `async def` declarations and instance methods. A decorator is an ordinary callable value that receives the decorated function value and returns the binding that should replace it: + r#"User-defined decorators are valid on top-level `def` / `async def` declarations and instance methods. A decorator is an ordinary callable value that receives the decorated function or method callable and returns the callable that should replace it: ```incan def parse(value: int) -> int: @@ -500,6 +500,45 @@ def main() -> None: Stacked decorators apply bottom-up, matching Python's declaration model: the decorator closest to `def` receives the original function value first, and the outer decorators receive each previous result. Decorator factories such as `@logged("name")` are checked by first evaluating the factory expression as a callable-producing expression and then applying the produced decorator to the function value. +!!! tip "Coming from Python?" + Python decorators can replace a function with any object. Incan user-defined function decorators are stricter: the decorator input is the decorated callable, and the result must also be callable. Python's `Callable[[A, B], R]` corresponds to Incan's `(A, B) -> R`; `=>` is only for closure expressions, not callable types. Use `(F) -> F` when a decorator preserves the original callable signature, and spell the source and replacement callable types separately when it intentionally changes the signature, such as `((str) -> R) -> ((str, str) -> R)`. + +Decorator factories can be generic over the decorated function type. This is the usual shape for registry, catalog, routing, telemetry, and validation decorators that record metadata but return the original function unchanged: + +```incan +def registered[F](function_ref: str) -> ((F) -> F): + return (func) => func + +@registered("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +The compiler infers `F` from the decorated function when the factory result is applied. If inference needs help, pass the decorated function type explicitly on the decorator factory call: + +```incan +@registered[(str) -> ColumnExpr]("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +The post-decoration binding keeps the concrete callable signature of the decorated function unless the decorator deliberately returns a different callable shape. Checked API metadata and imports observe that concrete signature, not the generic helper's `F`. + +The callable value passed into a decorator exposes `__name__` as the source callable name. Registry and catalog decorators can use this from concrete decorator helpers and from generic `(F) -> F` helpers, so a decorator can record `func.__name__` without requiring the decorated declaration to repeat its own public name in a string argument. + +```incan +def capture[F](func: F) -> F: + registry_names.append(func.__name__) + return func + +def registered[F]() -> ((F) -> F): + return (func) => capture[F](func) + +@registered() +pub def sample(value: int) -> int: + return value + 1 +``` + Method decorators receive an unbound callable shape with the receiver first. A decorator on `def label(self, value: int) -> str` sees `(&Box, int) -> str`; a decorator on `def bump(mut self, value: int) -> int` sees `(&mut Box, int) -> int`. The wrapper passes the actual receiver borrow through to the decorated callable, so method decorators do not require cloning the receiver. Class, model, trait, enum, newtype, field, alias, and module decorators remain limited to compiler-owned decorators. Compiler-owned decorators such as `@derive`, `@route`, `@rust.extern`, `@rust.allow`, `@staticmethod`, `@classmethod`, and `@requires` keep their existing special behavior. @@ -1065,6 +1104,31 @@ fn render_surface_methods_section(out: &mut String) { } out.push('\n'); + // Iterator + out.push_str("\n### Iterator methods\n\n"); + out.push_str(table_header()); + for m in surface::iterator_methods::ITERATOR_METHODS { + let id = format!("{:?}", m.id); + let canonical = format!("`{}`", m.canonical); + let aliases = if m.aliases.is_empty() { + String::new() + } else { + m.aliases + .iter() + .map(|a| format!("`{}`", a)) + .collect::>() + .join(", ") + }; + let desc = m.description; + let rfc = m.introduced_in_rfc; + let since = m.since; + let stability = format!("{:?}", m.stability); + out.push_str(&format!( + "| {id} | {canonical} | {aliases} | {desc} | {rfc} | {since} | {stability} |\n" + )); + } + out.push('\n'); + // Frozen containers out.push_str("\n### FrozenList methods\n\n"); out.push_str(table_header()); diff --git a/crates/incan_core/src/interop/extension_traits.rs b/crates/incan_core/src/interop/extension_traits.rs new file mode 100644 index 000000000..1e0aabbf1 --- /dev/null +++ b/crates/incan_core/src/interop/extension_traits.rs @@ -0,0 +1,66 @@ +//! Fallback Rust extension-trait method vocabulary used when rust-inspect metadata is unavailable. + +/// Return fallback trait method names for Rust traits when structured trait metadata is unavailable. +#[must_use] +pub fn fallback_rust_trait_methods(path: &str) -> &'static [&'static str] { + match path { + "std::io::Read" => &[ + "read", + "read_to_end", + "read_to_string", + "read_exact", + "read_buf", + "read_buf_exact", + "bytes", + "chain", + "take", + ], + "std::io::Write" => &["write", "write_all", "write_fmt", "flush"], + "std::io::Seek" => &["seek", "rewind", "stream_position", "seek_relative"], + "byteorder::ReadBytesExt" => &[ + "read_u8", + "read_i8", + "read_u16", + "read_i16", + "read_u32", + "read_i32", + "read_u64", + "read_i64", + "read_u128", + "read_i128", + "read_f32", + "read_f64", + ], + "byteorder::WriteBytesExt" => &[ + "write_u8", + "write_i8", + "write_u16", + "write_i16", + "write_u32", + "write_i32", + "write_u64", + "write_i64", + "write_u128", + "write_i128", + "write_f32", + "write_f64", + ], + "sha2::Digest" | "sha3::Digest" | "blake2::Digest" | "md5::Digest" | "sha1::Digest" => &[ + "new", + "new_with_prefix", + "update", + "chain_update", + "finalize", + "finalize_into", + "finalize_reset", + "reset", + "output_size", + "digest", + ], + "blake2::digest::XofReader" | "sha3::digest::XofReader" => &["read"], + "std::os::unix::fs::MetadataExt" => &[ + "dev", "ino", "mode", "nlink", "uid", "gid", "rdev", "size", "atime", "mtime", "ctime", "blksize", "blocks", + ], + _ => &[], + } +} diff --git a/crates/incan_core/src/interop/metadata.rs b/crates/incan_core/src/interop/metadata.rs index 8bf656974..92400b7d6 100644 --- a/crates/incan_core/src/interop/metadata.rs +++ b/crates/incan_core/src/interop/metadata.rs @@ -114,6 +114,132 @@ impl RustItemMetadata { } } +/// Borrow shape for a metadata-free external method compatibility policy. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum MetadataFreeMethodArgBorrowPolicy { + Shared, + Mutable, +} + +/// Receiver class used by metadata-free external method compatibility policy. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum MetadataFreeReceiverClass { + IoValue, + EncodingInstance, + ExternalAssociated, +} + +/// Argument class used by metadata-free external method compatibility policy. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum MetadataFreeArgClass { + StringBuffer, + ByteBuffer, + Any, +} + +/// Borrow compatibility rule for one metadata-free Rust method surface. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MetadataFreeMethodBorrowRule { + pub methods: &'static [&'static str], + pub receiver: MetadataFreeReceiverClass, + pub arg: MetadataFreeArgClass, + pub policy: MetadataFreeMethodArgBorrowPolicy, +} + +/// One parameter in a metadata-free Rust method signature. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MetadataFreeMethodParamRule { + pub name: Option<&'static str>, + pub type_display: &'static str, +} + +/// Complete callable signature for one metadata-free Rust method surface. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MetadataFreeMethodSignatureRule { + pub receiver_path: &'static str, + pub method: &'static str, + pub params: &'static [MetadataFreeMethodParamRule], + pub return_type: &'static str, + pub is_async: bool, + pub is_unsafe: bool, +} + +/// Metadata-free external method borrow policies used when rust-inspect metadata is unavailable. +pub const METADATA_FREE_METHOD_BORROW_RULES: &[MetadataFreeMethodBorrowRule] = &[ + MetadataFreeMethodBorrowRule { + methods: &["read_to_string"], + receiver: MetadataFreeReceiverClass::IoValue, + arg: MetadataFreeArgClass::StringBuffer, + policy: MetadataFreeMethodArgBorrowPolicy::Mutable, + }, + MetadataFreeMethodBorrowRule { + methods: &["read", "read_to_end", "read_exact", "read_buf", "read_buf_exact"], + receiver: MetadataFreeReceiverClass::IoValue, + arg: MetadataFreeArgClass::ByteBuffer, + policy: MetadataFreeMethodArgBorrowPolicy::Mutable, + }, + MetadataFreeMethodBorrowRule { + methods: &["write"], + receiver: MetadataFreeReceiverClass::IoValue, + arg: MetadataFreeArgClass::ByteBuffer, + policy: MetadataFreeMethodArgBorrowPolicy::Shared, + }, + MetadataFreeMethodBorrowRule { + methods: &["write_all"], + receiver: MetadataFreeReceiverClass::IoValue, + arg: MetadataFreeArgClass::Any, + policy: MetadataFreeMethodArgBorrowPolicy::Shared, + }, + MetadataFreeMethodBorrowRule { + methods: &["for_label", "encode", "decode"], + receiver: MetadataFreeReceiverClass::EncodingInstance, + arg: MetadataFreeArgClass::Any, + policy: MetadataFreeMethodArgBorrowPolicy::Shared, + }, + MetadataFreeMethodBorrowRule { + methods: &["decode"], + receiver: MetadataFreeReceiverClass::ExternalAssociated, + arg: MetadataFreeArgClass::ByteBuffer, + policy: MetadataFreeMethodArgBorrowPolicy::Shared, + }, +]; + +/// Metadata-free external method signatures used when rust-inspect metadata is unavailable. +pub const METADATA_FREE_METHOD_SIGNATURE_RULES: &[MetadataFreeMethodSignatureRule] = + &[MetadataFreeMethodSignatureRule { + receiver_path: "encoding_rs::Encoding", + method: "for_label", + params: &[MetadataFreeMethodParamRule { + name: Some("label"), + type_display: "&[u8]", + }], + return_type: "Option<&'static encoding_rs::Encoding>", + is_async: false, + is_unsafe: false, + }]; + +/// Return conservative callable metadata for Rust surfaces the stdlib must compile against even when rust-inspect +/// cannot recover full crate metadata in generated smoke projects. +#[must_use] +pub fn metadata_free_method_signature(rust_path: &str, method: &str) -> Option { + let rule = METADATA_FREE_METHOD_SIGNATURE_RULES + .iter() + .find(|rule| rule.receiver_path == rust_path && rule.method == method)?; + Some(RustFunctionSig { + params: rule + .params + .iter() + .map(|param| RustParam { + name: param.name.map(str::to_string), + type_display: param.type_display.to_string(), + }) + .collect(), + return_type: rule.return_type.to_string(), + is_async: rule.is_async, + is_unsafe: rule.is_unsafe, + }) +} + /// A single parameter in a Rust function signature (display strings only for Phase 1). #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct RustParam { @@ -177,6 +303,105 @@ pub enum RustTypeShape { Unknown, } +/// Render `path` with generic arguments as `path` for stable Rust-like display. +#[must_use] +pub fn render_rust_type_shape_path(path: &str, args: &[RustTypeShape]) -> String { + if args.is_empty() { + return path.to_string(); + } + let rendered_args: Vec = args.iter().map(render_rust_type_shape).collect(); + format!("{path}<{}>", rendered_args.join(", ")) +} + +/// Pretty-print a [`RustTypeShape`] as a stable Rust-like type string. +#[must_use] +pub fn render_rust_type_shape(shape: &RustTypeShape) -> String { + match shape { + RustTypeShape::Bool => "bool".to_string(), + RustTypeShape::Float => "f64".to_string(), + RustTypeShape::Int => "i64".to_string(), + RustTypeShape::Str => "String".to_string(), + RustTypeShape::Bytes => "Vec".to_string(), + RustTypeShape::Unit => "()".to_string(), + RustTypeShape::Option(inner) => format!("Option<{}>", render_rust_type_shape(inner)), + RustTypeShape::Result(ok, err) => { + format!( + "Result<{}, {}>", + render_rust_type_shape(ok), + render_rust_type_shape(err) + ) + } + RustTypeShape::Tuple(items) => { + let rendered: Vec = items.iter().map(render_rust_type_shape).collect(); + format!("({})", rendered.join(", ")) + } + RustTypeShape::Ref(inner) => format!("&{}", render_rust_type_shape(inner)), + RustTypeShape::RustPath { path, args } => render_rust_type_shape_path(path, args), + RustTypeShape::TypeParam(name) => name.clone(), + RustTypeShape::Unknown => "?".to_string(), + } +} + +/// Remove Rust lifetime labels that decorate borrowed display types. +#[must_use] +pub fn strip_rust_borrow_lifetimes(text: &str) -> String { + let mut out = String::with_capacity(text.len()); + let mut chars = text.chars().peekable(); + while let Some(ch) = chars.next() { + out.push(ch); + if ch != '&' { + continue; + } + while matches!(chars.peek(), Some(next) if next.is_whitespace()) { + if let Some(next) = chars.next() { + out.push(next); + } + } + if !matches!(chars.peek(), Some('\'')) { + continue; + } + chars.next(); + while matches!(chars.peek(), Some(next) if next.is_ascii_alphanumeric() || *next == '_') { + chars.next(); + } + while matches!(chars.peek(), Some(next) if next.is_whitespace()) { + chars.next(); + } + } + out +} + +/// Split a comma-separated Rust generic/tuple argument list without splitting inside nested generic, tuple, or slice +/// delimiters. +#[must_use] +pub fn split_top_level_rust_args(text: &str) -> Vec<&str> { + let mut args = Vec::new(); + let mut start = 0usize; + let mut angle = 0usize; + let mut paren = 0usize; + let mut bracket = 0usize; + for (idx, ch) in text.char_indices() { + match ch { + '<' => angle += 1, + '>' => angle = angle.saturating_sub(1), + '(' => paren += 1, + ')' => paren = paren.saturating_sub(1), + '[' => bracket += 1, + ']' => bracket = bracket.saturating_sub(1), + ',' if angle == 0 && paren == 0 && bracket == 0 => { + args.push(text[start..idx].trim()); + start = idx + ch.len_utf8(); + } + _ => {} + } + } + let tail = text[start..].trim(); + if !tail.is_empty() { + args.push(tail); + } + args +} + /// A public field surfaced on a Rust struct/union-like type. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct RustFieldInfo { diff --git a/crates/incan_core/src/interop/mod.rs b/crates/incan_core/src/interop/mod.rs index 0abb18740..69eaeafcf 100644 --- a/crates/incan_core/src/interop/mod.rs +++ b/crates/incan_core/src/interop/mod.rs @@ -6,12 +6,18 @@ pub mod capabilities; pub mod coercions; +mod extension_traits; pub mod metadata; pub use capabilities::{RUST_CAPABILITY_BOUNDS, is_rust_capability_bound}; pub use coercions::{CoercionPolicy, admitted_builtin_coercion}; +pub use extension_traits::fallback_rust_trait_methods; pub use metadata::{ - RustCollectionFamily, RustFieldInfo, RustFunctionSig, RustImplementedTrait, RustItemKind, RustItemMetadata, - RustMethodSig, RustModuleChild, RustModuleChildKind, RustModuleInfo, RustParam, RustTraitAssoc, RustTraitInfo, - RustTypeInfo, RustTypeShape, RustVariantInfo, RustVisibility, + METADATA_FREE_METHOD_BORROW_RULES, METADATA_FREE_METHOD_SIGNATURE_RULES, MetadataFreeArgClass, + MetadataFreeMethodArgBorrowPolicy, MetadataFreeMethodBorrowRule, MetadataFreeMethodParamRule, + MetadataFreeMethodSignatureRule, MetadataFreeReceiverClass, RustCollectionFamily, RustFieldInfo, RustFunctionSig, + RustImplementedTrait, RustItemKind, RustItemMetadata, RustMethodSig, RustModuleChild, RustModuleChildKind, + RustModuleInfo, RustParam, RustTraitAssoc, RustTraitInfo, RustTypeInfo, RustTypeShape, RustVariantInfo, + RustVisibility, metadata_free_method_signature, render_rust_type_shape, render_rust_type_shape_path, + split_top_level_rust_args, strip_rust_borrow_lifetimes, }; diff --git a/crates/incan_core/src/lang/features.rs b/crates/incan_core/src/lang/features.rs index e7ad82d54..efe27205a 100644 --- a/crates/incan_core/src/lang/features.rs +++ b/crates/incan_core/src/lang/features.rs @@ -500,8 +500,13 @@ pub const FEATURES: &[FeatureDescriptor] = &[ introduced_in_rfc: RFC::_036, stability: Stability::Stable, activation: "None for user-defined decorators; compiler-owned decorators keep their documented imports.", - summary: "Decorators are ordinary callable values applied to functions and methods, including decorator factories.", - canonical_forms: &["@logged", "@route(\"/users\")", "@trace(level=Level.INFO)"], + summary: "Decorators are ordinary callable values applied to functions and methods, including generic decorator factories that infer or accept the decorated function type and decorator helpers that expose `func.__name__`.", + canonical_forms: &[ + "@logged", + "@registered(\"catalog.ref\")", + "func.__name__", + "@registered[(str) -> ColumnExpr](\"catalog.ref\")", + ], prefer_over: "Boilerplate wrapper declarations around every function that needs the same callable transform.", references: links![ ("Language reference", "language.md#decorators"), diff --git a/crates/incan_core/src/lang/mod.rs b/crates/incan_core/src/lang/mod.rs index d97648563..73914e850 100644 --- a/crates/incan_core/src/lang/mod.rs +++ b/crates/incan_core/src/lang/mod.rs @@ -42,6 +42,7 @@ pub mod registry; pub mod rust_keywords; pub mod stdlib; pub mod surface; +pub mod testing; pub mod trait_bounds; pub mod trait_capabilities; pub mod traits; diff --git a/crates/incan_core/src/lang/stdlib.rs b/crates/incan_core/src/lang/stdlib.rs index 1a0bbeb4d..8bb1717a0 100644 --- a/crates/incan_core/src/lang/stdlib.rs +++ b/crates/incan_core/src/lang/stdlib.rs @@ -33,17 +33,93 @@ pub const STDLIB_RUST: &str = "rust"; pub const STDLIB_BUILTINS: &str = "builtins"; /// `std.json` module name. pub const STDLIB_JSON: &str = "json"; +/// `std.serde` module name. +pub const STDLIB_SERDE: &str = "serde"; /// Dynamic JSON value type exported by `std.json`. pub const JSON_VALUE_TYPE_NAME: &str = "JsonValue"; /// Runtime Rust path carried by `std.json.JsonValue`. pub const JSON_VALUE_RUST_PATH: &str = "incan_stdlib::json::JsonValue"; +/// Stable ids for compiler-known stdlib JSON protocol traits. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum StdlibJsonTraitId { + Serialize, + Deserialize, +} + +const STDLIB_JSON_SERIALIZE_TRAIT_NAMES: &[&str] = &[ + "Serialize", + "JsonSerialize", + "json.Serialize", + "std.serde.json.Serialize", +]; + +const STDLIB_JSON_DESERIALIZE_TRAIT_NAMES: &[&str] = &[ + "Deserialize", + "JsonDeserialize", + "json.Deserialize", + "std.serde.json.Deserialize", +]; + /// Return whether `name` is the canonical dynamic JSON value type. #[must_use] pub fn is_json_value_type_name(name: &str) -> bool { name == JSON_VALUE_TYPE_NAME } +/// Return the stdlib JSON trait id for a source, alias, or qualified trait spelling. +#[must_use] +pub fn stdlib_json_trait_id(name: &str) -> Option { + if STDLIB_JSON_SERIALIZE_TRAIT_NAMES.contains(&name) { + Some(StdlibJsonTraitId::Serialize) + } else if STDLIB_JSON_DESERIALIZE_TRAIT_NAMES.contains(&name) { + Some(StdlibJsonTraitId::Deserialize) + } else { + None + } +} + +/// Return whether `segments` names the `std.serde.json` trait module. +#[must_use] +pub fn is_stdlib_json_trait_module_path(segments: &[String]) -> bool { + matches!( + segments, + [std, serde, json] + if std == STDLIB_ROOT && serde == STDLIB_SERDE && json == STDLIB_JSON + ) +} + +/// Return the stdlib JSON trait id for a resolved source import path. +#[must_use] +pub fn stdlib_json_trait_id_from_path(segments: &[String]) -> Option { + if is_stdlib_json_trait_module_path(segments) { + return None; + } + stdlib_json_trait_id(&segments.join(".")) +} + +/// Return the stdlib JSON trait id when generated Rust must import the trait module for method resolution. +#[must_use] +pub fn stdlib_json_trait_scope_import_id(name: &str) -> Option { + match name { + "json.Serialize" | "std.serde.json.Serialize" => Some(StdlibJsonTraitId::Serialize), + "json.Deserialize" | "std.serde.json.Deserialize" => Some(StdlibJsonTraitId::Deserialize), + _ => None, + } +} + +/// Return whether `name` refers to the stdlib JSON serialization trait. +#[must_use] +pub fn is_stdlib_json_serialize_trait_name(name: &str) -> bool { + stdlib_json_trait_id(name) == Some(StdlibJsonTraitId::Serialize) +} + +/// Return whether `name` refers to the stdlib JSON deserialization trait. +#[must_use] +pub fn is_stdlib_json_deserialize_trait_name(name: &str) -> bool { + stdlib_json_trait_id(name) == Some(StdlibJsonTraitId::Deserialize) +} + const STDLIB_GRAPH_CONSTRUCTOR_TYPES: &[&str] = &["DiGraph", "Dag", "MultiDiGraph"]; /// Check if a module path starts with `std.`. @@ -96,6 +172,8 @@ pub struct StdlibExtraCrateDep { pub crate_name: &'static str, /// Dependency source and version/path metadata. pub source: StdlibExtraCrateSource, + /// Cargo features enabled for this stdlib-managed dependency. + pub features: &'static [&'static str], } /// Source descriptor for a namespace-provided extra crate dependency. @@ -204,14 +282,17 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ StdlibExtraCrateDep { crate_name: "incan_web_macros", source: StdlibExtraCrateSource::Path("crates/incan_web_macros"), + features: &[], }, StdlibExtraCrateDep { crate_name: "inventory", source: StdlibExtraCrateSource::Version("0.3"), + features: &[], }, StdlibExtraCrateDep { crate_name: "axum", source: StdlibExtraCrateSource::Version("0.8"), + features: &[], }, ], submodules: &["app", "routing", "request", "response", "macros", "prelude"], @@ -248,14 +329,22 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ StdlibNamespace { name: "serde", feature: Some("json"), - extra_crate_deps: &[], + extra_crate_deps: &[StdlibExtraCrateDep { + crate_name: "serde", + source: StdlibExtraCrateSource::Version("1.0"), + features: &["derive"], + }], submodules: &["json"], typechecker_only: false, }, StdlibNamespace { name: STDLIB_JSON, feature: Some("json"), - extra_crate_deps: &[], + extra_crate_deps: &[StdlibExtraCrateDep { + crate_name: "serde", + source: StdlibExtraCrateSource::Version("1.0"), + features: &["derive"], + }], submodules: &[], typechecker_only: false, }, @@ -293,6 +382,7 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ extra_crate_deps: &[StdlibExtraCrateDep { crate_name: "libm", source: StdlibExtraCrateSource::Version("0.2"), + features: &[], }], submodules: &[], typechecker_only: false, @@ -332,6 +422,7 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ extra_crate_deps: &[StdlibExtraCrateDep { crate_name: "rand", source: StdlibExtraCrateSource::Version("0.8"), + features: &[], }], submodules: &[], typechecker_only: false, @@ -342,6 +433,7 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ extra_crate_deps: &[StdlibExtraCrateDep { crate_name: "regex", source: StdlibExtraCrateSource::Version("1.0"), + features: &[], }], submodules: &["_core", "_replacement", "types", "prelude"], typechecker_only: false, @@ -359,6 +451,7 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ extra_crate_deps: &[StdlibExtraCrateDep { crate_name: "byteorder", source: StdlibExtraCrateSource::Version("1"), + features: &[], }], submodules: &[], typechecker_only: false, @@ -375,7 +468,43 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ StdlibNamespace { name: "hash", feature: None, - extra_crate_deps: &[], + extra_crate_deps: &[ + StdlibExtraCrateDep { + crate_name: "blake2", + source: StdlibExtraCrateSource::Version("0.10"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "blake3", + source: StdlibExtraCrateSource::Version("1"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "md5", + source: StdlibExtraCrateSource::Version("0.10"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "sha1", + source: StdlibExtraCrateSource::Version("0.10"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "sha2", + source: StdlibExtraCrateSource::Version("0.10"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "sha3", + source: StdlibExtraCrateSource::Version("0.10"), + features: &[], + }, + StdlibExtraCrateDep { + crate_name: "xxhash_rust", + source: StdlibExtraCrateSource::Version("0.8"), + features: &["xxh3", "xxh32", "xxh64"], + }, + ], submodules: &["_core", "_streaming", "prelude"], typechecker_only: false, }, @@ -386,22 +515,27 @@ pub const STDLIB_NAMESPACES: &[StdlibNamespace] = &[ StdlibExtraCrateDep { crate_name: "flate2", source: StdlibExtraCrateSource::Version("1"), + features: &[], }, StdlibExtraCrateDep { crate_name: "zstd", source: StdlibExtraCrateSource::Version("0.13"), + features: &[], }, StdlibExtraCrateDep { crate_name: "bzip2", source: StdlibExtraCrateSource::Version("0.6"), + features: &[], }, StdlibExtraCrateDep { crate_name: "xz2", source: StdlibExtraCrateSource::Version("0.1"), + features: &[], }, StdlibExtraCrateDep { crate_name: "snap", source: StdlibExtraCrateSource::Version("1"), + features: &[], }, ], submodules: &[ @@ -450,6 +584,39 @@ pub fn find_namespace(name: &str) -> Option<&'static StdlibNamespace> { STDLIB_NAMESPACES.iter().find(|ns| ns.name == name) } +/// Look up an extra Cargo crate dependency declared by any registered stdlib namespace. +/// +/// This is the registry boundary for compiler subsystems that need stdlib-managed dependency metadata without +/// duplicating namespace traversal or crate version knowledge. +#[must_use] +pub fn find_extra_crate_dep(crate_name: &str) -> Option<&'static StdlibExtraCrateDep> { + extra_crate_deps().find(|dep| dep.crate_name == crate_name) +} + +/// Return whether a crate is supplied by the workspace as a stdlib-managed path dependency. +#[must_use] +pub fn is_path_extra_crate_dep(crate_name: &str) -> bool { + find_extra_crate_dep(crate_name).is_some_and(|dep| matches!(dep.source, StdlibExtraCrateSource::Path(_))) +} + +/// Return the published Cargo package name when a stdlib-managed Rust crate imports under a different crate key. +#[must_use] +pub fn extra_crate_package_alias(crate_name: &str) -> Option<&'static str> { + match crate_name { + "md5" => Some("md-5"), + "xxhash_rust" => Some("xxhash-rust"), + _ => None, + } +} + +/// Iterate over every extra Cargo crate dependency declared by registered stdlib namespaces. +/// +/// Consumers that need to filter by dependency source can use this iterator while keeping namespace traversal +/// centralized in the stdlib registry. +pub fn extra_crate_deps() -> impl Iterator { + STDLIB_NAMESPACES.iter().flat_map(|ns| ns.extra_crate_deps) +} + /// Return the stdlib module path that owns fallback method signatures for a builtin trait name. /// /// The returned segments can be passed to the typechecker's stdlib cache to load the full `.incn` trait declaration @@ -829,6 +996,66 @@ mod tests { assert_eq!(trait_method_module_segments("Serialize"), None); } + #[test] + fn stdlib_json_trait_lookup_covers_aliases_and_qualified_names() { + for name in [ + "Serialize", + "JsonSerialize", + "json.Serialize", + "std.serde.json.Serialize", + ] { + assert_eq!(stdlib_json_trait_id(name), Some(StdlibJsonTraitId::Serialize)); + assert!(is_stdlib_json_serialize_trait_name(name)); + } + + for name in [ + "Deserialize", + "JsonDeserialize", + "json.Deserialize", + "std.serde.json.Deserialize", + ] { + assert_eq!(stdlib_json_trait_id(name), Some(StdlibJsonTraitId::Deserialize)); + assert!(is_stdlib_json_deserialize_trait_name(name)); + } + + assert_eq!(stdlib_json_trait_id("yaml.Serialize"), None); + assert_eq!(stdlib_json_trait_scope_import_id("Serialize"), None); + assert_eq!(stdlib_json_trait_scope_import_id("JsonSerialize"), None); + assert_eq!( + stdlib_json_trait_scope_import_id("json.Serialize"), + Some(StdlibJsonTraitId::Serialize) + ); + let json_trait_module = vec!["std".to_string(), "serde".to_string(), "json".to_string()]; + assert!(is_stdlib_json_trait_module_path(&json_trait_module)); + let serialize_path = vec![ + "std".to_string(), + "serde".to_string(), + "json".to_string(), + "Serialize".to_string(), + ]; + assert_eq!( + stdlib_json_trait_id_from_path(&serialize_path), + Some(StdlibJsonTraitId::Serialize) + ); + } + + #[test] + fn extra_crate_dependency_lookup_is_registry_driven() { + let axum = find_extra_crate_dep("axum"); + assert_eq!(axum.map(|dep| dep.crate_name), Some("axum")); + assert_eq!(axum.map(|dep| dep.source), Some(StdlibExtraCrateSource::Version("0.8"))); + + let macros = find_extra_crate_dep("incan_web_macros"); + assert_eq!( + macros.map(|dep| dep.source), + Some(StdlibExtraCrateSource::Path("crates/incan_web_macros")) + ); + assert!(is_path_extra_crate_dep("incan_web_macros")); + assert!(!is_path_extra_crate_dep("axum")); + + assert!(find_extra_crate_dep("not_a_stdlib_dependency").is_none()); + } + #[test] fn stdlib_registry_keeps_phase_023_metadata() { let async_ns = find_namespace("async"); @@ -839,6 +1066,8 @@ mod tests { let math_ns = find_namespace("math"); let graph_ns = find_namespace("graph"); let uuid_ns = find_namespace("uuid"); + let serde_ns = find_namespace("serde"); + let json_ns = find_namespace(STDLIB_JSON); let hash_ns = find_namespace("hash"); let datetime_ns = find_namespace("datetime"); let collections_ns = find_namespace("collections"); @@ -866,6 +1095,20 @@ mod tests { ); assert_eq!(uuid_ns.map(|ns| ns.submodules.is_empty()), Some(true)); assert_eq!(uuid_ns.map(|ns| ns.typechecker_only), Some(false)); + assert_eq!( + serde_ns.map(|ns| ns.extra_crate_deps.iter().map(|dep| dep.crate_name).collect::>()), + Some(vec!["serde"]) + ); + assert_eq!( + serde_ns + .and_then(|ns| ns.extra_crate_deps.first()) + .map(|dep| dep.features), + Some(&["derive"][..]) + ); + assert_eq!( + json_ns.map(|ns| ns.extra_crate_deps.iter().map(|dep| dep.crate_name).collect::>()), + Some(vec!["serde"]) + ); assert_eq!(collections_ns.map(|ns| ns.feature), Some(None)); assert_eq!(collections_ns.map(|ns| ns.extra_crate_deps.is_empty()), Some(true)); assert_eq!(collections_ns.map(|ns| ns.submodules.is_empty()), Some(true)); @@ -877,7 +1120,10 @@ mod tests { Some("byteorder") ); assert_eq!(hash_ns.map(|ns| ns.feature), Some(None)); - assert_eq!(hash_ns.map(|ns| ns.extra_crate_deps.is_empty()), Some(true)); + assert_eq!( + hash_ns.map(|ns| ns.extra_crate_deps.iter().map(|dep| dep.crate_name).collect::>()), + Some(vec!["blake2", "blake3", "md5", "sha1", "sha2", "sha3", "xxhash_rust",]) + ); assert_eq!(hash_ns.map(|ns| ns.submodules.contains(&"prelude")), Some(true)); assert_eq!(hash_ns.map(|ns| ns.submodules.contains(&"_core")), Some(true)); assert_eq!(hash_ns.map(|ns| ns.submodules.contains(&"_streaming")), Some(true)); diff --git a/crates/incan_core/src/lang/surface/methods.rs b/crates/incan_core/src/lang/surface/methods.rs index a1b2cd292..4664c21fc 100644 --- a/crates/incan_core/src/lang/surface/methods.rs +++ b/crates/incan_core/src/lang/surface/methods.rs @@ -1100,6 +1100,8 @@ pub mod result_methods { OrElse, Inspect, InspectErr, + Unwrap, + UnwrapOr, } pub type ResultMethodInfo = LangItemInfo; @@ -1153,6 +1155,22 @@ pub mod result_methods { RFC::_070, Since(0, 3), ), + info( + ResultMethodId::Unwrap, + "unwrap", + &[], + "Return the Ok payload or panic.", + RFC::_000, + Since(0, 1), + ), + info( + ResultMethodId::UnwrapOr, + "unwrap_or", + &[], + "Return the Ok payload or a default value.", + RFC::_000, + Since(0, 1), + ), ]; /// Resolve a result method spelling to its stable id. @@ -1194,3 +1212,157 @@ pub mod result_methods { } } } + +pub mod iterator_methods { + //! Iterator protocol method surface vocabulary. + + use super::LangItemInfo; + use crate::lang::registry::{RFC, Since, Stability}; + + /// Stable identifier for an RFC 088 iterator protocol method. + #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] + pub enum IteratorMethodId { + Iter, + Map, + Filter, + Enumerate, + Zip, + Take, + Skip, + TakeWhile, + SkipWhile, + Chain, + FlatMap, + Batch, + Collect, + Count, + Reduce, + Fold, + Any, + All, + Find, + ForEach, + Sum, + } + + pub type IteratorMethodInfo = LangItemInfo; + + pub const ITERATOR_METHODS: &[IteratorMethodInfo] = &[ + info(IteratorMethodId::Iter, "iter", "Create an iterator over an iterable."), + info(IteratorMethodId::Map, "map", "Lazily transform iterator items."), + info( + IteratorMethodId::Filter, + "filter", + "Lazily keep items that match a predicate.", + ), + info( + IteratorMethodId::Enumerate, + "enumerate", + "Yield each item with its zero-based index.", + ), + info(IteratorMethodId::Zip, "zip", "Pair items from two iterables."), + info( + IteratorMethodId::Take, + "take", + "Yield at most the requested number of items.", + ), + info( + IteratorMethodId::Skip, + "skip", + "Discard at most the requested number of items.", + ), + info( + IteratorMethodId::TakeWhile, + "take_while", + "Yield items until a predicate first returns false.", + ), + info( + IteratorMethodId::SkipWhile, + "skip_while", + "Discard items while a predicate returns true.", + ), + info( + IteratorMethodId::Chain, + "chain", + "Yield receiver items followed by another iterable.", + ), + info( + IteratorMethodId::FlatMap, + "flat_map", + "Map items to iterables and flatten the result.", + ), + info(IteratorMethodId::Batch, "batch", "Yield fixed-size list batches."), + info(IteratorMethodId::Collect, "collect", "Consume an iterator into a list."), + info( + IteratorMethodId::Count, + "count", + "Consume an iterator and return the item count.", + ), + info( + IteratorMethodId::Reduce, + "reduce", + "Consume an iterator with an explicit accumulator.", + ), + info( + IteratorMethodId::Fold, + "fold", + "Consume an iterator with an explicit accumulator.", + ), + info( + IteratorMethodId::Any, + "any", + "Return whether any item satisfies a predicate.", + ), + info( + IteratorMethodId::All, + "all", + "Return whether every item satisfies a predicate.", + ), + info( + IteratorMethodId::Find, + "find", + "Return the first item satisfying a predicate.", + ), + info( + IteratorMethodId::ForEach, + "for_each", + "Consume an iterator for side effects.", + ), + info( + IteratorMethodId::Sum, + "sum", + "Consume an iterator and return the numeric sum.", + ), + ]; + + /// Resolve an iterator method spelling to its stable id. + pub fn from_str(name: &str) -> Option { + super::from_str_impl(ITERATOR_METHODS, name) + } + + /// Return the canonical spelling for an iterator method. + pub fn as_str(id: IteratorMethodId) -> &'static str { + info_for(id).canonical + } + + /// Return the full metadata entry for an iterator method. + /// + /// ## Panics + /// - If the registry is missing an entry for `id` (this indicates a programming error). + pub fn info_for(id: IteratorMethodId) -> &'static IteratorMethodInfo { + super::info_for_impl(ITERATOR_METHODS, id, "iterator method info missing") + } + + const fn info(id: IteratorMethodId, canonical: &'static str, description: &'static str) -> IteratorMethodInfo { + LangItemInfo { + id, + canonical, + aliases: &[], + description, + introduced_in_rfc: RFC::_088, + since: Since(0, 3), + stability: Stability::Stable, + examples: &[], + } + } +} diff --git a/crates/incan_core/src/lang/surface/mod.rs b/crates/incan_core/src/lang/surface/mod.rs index 90e84d63a..f85b30c5f 100644 --- a/crates/incan_core/src/lang/surface/mod.rs +++ b/crates/incan_core/src/lang/surface/mod.rs @@ -20,5 +20,5 @@ pub mod types; // `crate::lang::surface::string_methods`, `crate::lang::surface::list_methods`, ... pub use methods::{ dict_methods, float_methods, frozen_bytes_methods, frozen_dict_methods, frozen_list_methods, frozen_set_methods, - list_methods, option_methods, result_methods, set_methods, string_methods, + iterator_methods, list_methods, option_methods, result_methods, set_methods, string_methods, }; diff --git a/crates/incan_core/src/lang/testing.rs b/crates/incan_core/src/lang/testing.rs new file mode 100644 index 000000000..081bfd946 --- /dev/null +++ b/crates/incan_core/src/lang/testing.rs @@ -0,0 +1,158 @@ +//! Shared testing marker vocabulary. + +use super::registry::{LangItemInfo, RFC, Since, Stability}; +use super::stdlib; + +/// Standard-library testing module segment. +pub const STDLIB_TESTING_MODULE: &str = "testing"; + +/// Stable identifier for a canonical `std.testing` assertion helper. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum TestingAssertHelperId { + Assert, + AssertFalse, + AssertEq, + AssertNe, + AssertIsSome, + AssertIsNone, + AssertIsOk, + AssertIsErr, + AssertRaises, +} + +pub type TestingAssertHelperInfo = LangItemInfo; + +/// Canonical `std.testing` assertion helpers with compiler-specialized emission. +pub const TESTING_ASSERT_HELPERS: &[TestingAssertHelperInfo] = &[ + assert_helper(TestingAssertHelperId::Assert, "assert"), + assert_helper(TestingAssertHelperId::AssertFalse, "assert_false"), + assert_helper(TestingAssertHelperId::AssertEq, "assert_eq"), + assert_helper(TestingAssertHelperId::AssertNe, "assert_ne"), + assert_helper(TestingAssertHelperId::AssertIsSome, "assert_is_some"), + assert_helper(TestingAssertHelperId::AssertIsNone, "assert_is_none"), + assert_helper(TestingAssertHelperId::AssertIsOk, "assert_is_ok"), + assert_helper(TestingAssertHelperId::AssertIsErr, "assert_is_err"), + assert_helper(TestingAssertHelperId::AssertRaises, "assert_raises"), +]; + +/// Resolve an assertion helper spelling to its stable id. +pub fn assert_helper_from_str(name: &str) -> Option { + TESTING_ASSERT_HELPERS + .iter() + .find(|helper| helper.canonical == name) + .map(|helper| helper.id) +} + +/// Return the canonical spelling for an assertion helper id. +/// +/// ## Panics +/// - If the registry is missing an entry for `id` (this indicates a programming error). +pub fn assert_helper_as_str(id: TestingAssertHelperId) -> &'static str { + TESTING_ASSERT_HELPERS + .iter() + .find(|helper| helper.id == id) + .unwrap_or_else(|| panic!("testing assert helper info missing")) + .canonical +} + +/// Return the canonical fully qualified `std.testing` path for an assertion helper. +#[must_use] +pub fn assert_helper_path(id: TestingAssertHelperId) -> [&'static str; 3] { + [stdlib::STDLIB_ROOT, STDLIB_TESTING_MODULE, assert_helper_as_str(id)] +} + +/// Resolve a fully qualified `std.testing` path to an assertion helper id. +#[must_use] +pub fn assert_helper_id_from_std_path(path: &[String]) -> Option { + let [root, module, name] = path else { + return None; + }; + if root == stdlib::STDLIB_ROOT && module == STDLIB_TESTING_MODULE { + assert_helper_from_str(name) + } else { + None + } +} + +/// Return whether a fully qualified path names one specific `std.testing` assertion helper. +#[must_use] +pub fn is_assert_helper_std_path(path: &[String], id: TestingAssertHelperId) -> bool { + assert_helper_id_from_std_path(path) == Some(id) +} + +/// Return the default assertion failure text for helpers whose message does not depend on operands. +#[must_use] +pub fn assert_helper_default_failure_message(id: TestingAssertHelperId) -> Option<&'static str> { + match id { + TestingAssertHelperId::Assert | TestingAssertHelperId::AssertFalse => Some("AssertionError"), + TestingAssertHelperId::AssertIsSome => Some("AssertionError: expected Some, got None"), + TestingAssertHelperId::AssertIsNone => Some("AssertionError: expected None, got Some"), + TestingAssertHelperId::AssertIsOk => Some("AssertionError: expected Ok, got Err"), + TestingAssertHelperId::AssertIsErr => Some("AssertionError: expected Err, got Ok"), + TestingAssertHelperId::AssertEq | TestingAssertHelperId::AssertNe | TestingAssertHelperId::AssertRaises => None, + } +} + +/// Return the operand relation text used by comparison assertion failures. +#[must_use] +pub fn assert_comparison_failure_kind(id: TestingAssertHelperId) -> Option<&'static str> { + match id { + TestingAssertHelperId::AssertEq => Some("left != right"), + TestingAssertHelperId::AssertNe => Some("left == right"), + _ => None, + } +} + +const fn assert_helper(id: TestingAssertHelperId, canonical: &'static str) -> TestingAssertHelperInfo { + LangItemInfo { + id, + canonical, + aliases: &[], + description: "Canonical testing assertion helper.", + introduced_in_rfc: RFC::_018, + since: Since(0, 1), + stability: Stability::Stable, + examples: &[], + } +} + +/// Runtime marker name for `std.testing.test`. +pub const TESTING_MARKER_TEST: &str = "test"; +/// Runtime marker name for `std.testing.fixture`. +pub const TESTING_MARKER_FIXTURE: &str = "fixture"; +/// Runtime marker name for `std.testing.skip`. +pub const TESTING_MARKER_SKIP: &str = "skip"; +/// Runtime marker name for `std.testing.skipif`. +pub const TESTING_MARKER_SKIPIF: &str = "skipif"; +/// Runtime marker name for `std.testing.xfail`. +pub const TESTING_MARKER_XFAIL: &str = "xfail"; +/// Runtime marker name for `std.testing.xfailif`. +pub const TESTING_MARKER_XFAILIF: &str = "xfailif"; +/// Runtime marker name for `std.testing.slow`. +pub const TESTING_MARKER_SLOW: &str = "slow"; +/// Runtime marker name for `std.testing.mark`. +pub const TESTING_MARKER_MARK: &str = "mark"; +/// Runtime marker name for `std.testing.resource`. +pub const TESTING_MARKER_RESOURCE: &str = "resource"; +/// Runtime marker name for `std.testing.serial`. +pub const TESTING_MARKER_SERIAL: &str = "serial"; +/// Runtime marker name for `std.testing.timeout`. +pub const TESTING_MARKER_TIMEOUT: &str = "timeout"; +/// Runtime marker name for `std.testing.parametrize`. +pub const TESTING_MARKER_PARAMETRIZE: &str = "parametrize"; + +/// Runner-only marker names that must have matching `@rust.extern` metadata in `stdlib/testing.incn`. +pub const RUNNER_ONLY_MARKER_NAMES: &[&str] = &[ + TESTING_MARKER_TEST, + TESTING_MARKER_FIXTURE, + TESTING_MARKER_SKIP, + TESTING_MARKER_SKIPIF, + TESTING_MARKER_XFAIL, + TESTING_MARKER_XFAILIF, + TESTING_MARKER_SLOW, + TESTING_MARKER_MARK, + TESTING_MARKER_RESOURCE, + TESTING_MARKER_SERIAL, + TESTING_MARKER_TIMEOUT, + TESTING_MARKER_PARAMETRIZE, +]; diff --git a/crates/incan_core/src/lang/trait_bounds.rs b/crates/incan_core/src/lang/trait_bounds.rs index 7fdac0437..763bb80fc 100644 --- a/crates/incan_core/src/lang/trait_bounds.rs +++ b/crates/incan_core/src/lang/trait_bounds.rs @@ -126,6 +126,7 @@ pub mod rust { pub const CLONE: &str = "Clone"; // Formatting + pub const DEBUG: &str = "std::fmt::Debug"; pub const DISPLAY: &str = "std::fmt::Display"; // Arithmetic ops diff --git a/crates/incan_core/src/lang/types/collections.rs b/crates/incan_core/src/lang/types/collections.rs index 766d8172e..8f5813638 100644 --- a/crates/incan_core/src/lang/types/collections.rs +++ b/crates/incan_core/src/lang/types/collections.rs @@ -155,6 +155,17 @@ pub fn from_str(name: &str) -> Option { .map(|t| t.id) } +/// Resolve a Rust generic display base such as `Vec`, `HashMap`, or `HashSet` into the matching +/// Incan collection type without making Rust-specific names valid source-level aliases. +pub fn from_rust_display_base(base: &str) -> Option { + let tail = base.rsplit("::").next().unwrap_or(base); + match tail { + "HashMap" => Some(CollectionTypeId::Dict), + "HashSet" => Some(CollectionTypeId::Set), + _ => from_str(tail), + } +} + /// Return the canonical spelling for a collection/generic-base builtin type. /// /// ## Parameters diff --git a/crates/incan_core/tests/lang_registry_guardrails.rs b/crates/incan_core/tests/lang_registry_guardrails.rs index bc87df9c0..6f789032e 100644 --- a/crates/incan_core/tests/lang_registry_guardrails.rs +++ b/crates/incan_core/tests/lang_registry_guardrails.rs @@ -10,7 +10,8 @@ use incan_core::lang::operators; use incan_core::lang::punctuation; use incan_core::lang::registry::{RFC, Since}; use incan_core::lang::surface::types::{SurfaceTypeCategory, SurfaceTypeId, SurfaceTypeOwner}; -use incan_core::lang::surface::{constructors, functions, types as surface_types}; +use incan_core::lang::surface::{constructors, functions, iterator_methods, result_methods, types as surface_types}; +use incan_core::lang::testing; use incan_core::lang::traits; use incan_core::lang::types::{collections, numerics, stringlike}; use std::path::{Path, PathBuf}; @@ -232,6 +233,19 @@ fn types_spellings_unique_and_resolvable() { }); } +#[test] +fn collection_rust_display_bases_are_not_ordinary_source_aliases() { + assert_eq!( + collections::from_rust_display_base("std::collections::HashSet"), + Some(collections::CollectionTypeId::Set) + ); + assert_eq!( + collections::from_rust_display_base("HashMap"), + Some(collections::CollectionTypeId::Dict) + ); + assert_eq!(collections::from_str("HashSet"), None); +} + #[test] fn derives_spellings_unique_and_resolvable() { assert_registry_round_trip(RegistryRoundTrip { @@ -342,6 +356,48 @@ fn surface_functions_spellings_unique_and_resolvable() { }); } +#[test] +fn iterator_methods_spellings_unique_and_resolvable() { + assert_registry_round_trip(RegistryRoundTrip { + label: "iterator method", + expected_len: 21, + items: iterator_methods::ITERATOR_METHODS, + id_of: |info| info.id, + canonical_of: |info| info.canonical, + aliases_of: |info| info.aliases, + from_str: iterator_methods::from_str, + as_str: iterator_methods::as_str, + }); +} + +#[test] +fn result_methods_spellings_unique_and_resolvable() { + assert_registry_round_trip(RegistryRoundTrip { + label: "result method", + expected_len: 8, + items: result_methods::RESULT_METHODS, + id_of: |info| info.id, + canonical_of: |info| info.canonical, + aliases_of: |info| info.aliases, + from_str: result_methods::from_str, + as_str: result_methods::as_str, + }); +} + +#[test] +fn testing_assert_helpers_spellings_unique_and_resolvable() { + assert_registry_round_trip(RegistryRoundTrip { + label: "testing assert helper", + expected_len: 9, + items: testing::TESTING_ASSERT_HELPERS, + id_of: |info| info.id, + canonical_of: |info| info.canonical, + aliases_of: |info| info.aliases, + from_str: testing::assert_helper_from_str, + as_str: testing::assert_helper_as_str, + }); +} + #[test] fn surface_types_spellings_unique_and_resolvable() { assert_registry_round_trip(RegistryRoundTrip { diff --git a/crates/incan_stdlib/src/testing.rs b/crates/incan_stdlib/src/testing.rs index 9a203fb4f..4fbd07091 100644 --- a/crates/incan_stdlib/src/testing.rs +++ b/crates/incan_stdlib/src/testing.rs @@ -3,6 +3,12 @@ //! `crates/incan_stdlib/stdlib/testing.incn` is the source-of-truth surface API for `std.testing`. //! This Rust module implements only host-boundary functions referenced by `@rust.extern` declarations in `std.testing`. +pub use incan_core::lang::testing::{ + RUNNER_ONLY_MARKER_NAMES, TESTING_MARKER_FIXTURE, TESTING_MARKER_MARK, TESTING_MARKER_PARAMETRIZE, + TESTING_MARKER_RESOURCE, TESTING_MARKER_SERIAL, TESTING_MARKER_SKIP, TESTING_MARKER_SKIPIF, TESTING_MARKER_SLOW, + TESTING_MARKER_TEST, TESTING_MARKER_TIMEOUT, TESTING_MARKER_XFAIL, TESTING_MARKER_XFAILIF, +}; + /// Generic panic primitive used by `std.testing` helpers with non-`None` return types. /// /// # Panics @@ -12,45 +18,48 @@ pub fn fail_t(msg: String) -> T { crate::errors::__private::raise_runtime_misuse(&msg) } +/// Return the canonical runtime misuse message for a runner-only `std.testing` marker. +pub fn testing_marker_runtime_misuse_message(marker: &str) -> String { + format!("std.testing.{marker} is marker metadata for `incan test` and is not executable runtime logic") +} + fn marker_runtime_misuse(marker: &str) -> ! { - crate::errors::__private::raise_runtime_misuse(&format!( - "std.testing.{marker} is marker metadata for `incan test` and is not executable runtime logic" - )); + crate::errors::__private::raise_runtime_misuse(&testing_marker_runtime_misuse_message(marker)); } /// Marker runtime for `@std.testing.skip`. /// /// `incan test` handles skip semantics during test discovery. Calling this at runtime is a misuse. pub fn skip(_reason: String) { - marker_runtime_misuse("skip"); + marker_runtime_misuse(TESTING_MARKER_SKIP); } /// Marker runtime for `@std.testing.skipif`. /// /// `incan test` evaluates skipif conditions during discovery. Calling this at runtime is a misuse. pub fn skipif(_condition: bool, _reason: String) { - marker_runtime_misuse("skipif"); + marker_runtime_misuse(TESTING_MARKER_SKIPIF); } /// Marker runtime for `@std.testing.test`. /// /// `incan test` handles explicit test discovery. Calling this at runtime is a misuse. pub fn test() { - marker_runtime_misuse("test"); + marker_runtime_misuse(TESTING_MARKER_TEST); } /// Marker runtime for `@std.testing.xfail`. /// /// `incan test` handles xfail semantics during test discovery/execution. Calling this at runtime is a misuse. pub fn xfail(_reason: String) { - marker_runtime_misuse("xfail"); + marker_runtime_misuse(TESTING_MARKER_XFAIL); } /// Marker runtime for `@std.testing.xfailif`. /// /// `incan test` evaluates xfailif conditions during discovery. Calling this at runtime is a misuse. pub fn xfailif(_condition: bool, _reason: String) { - marker_runtime_misuse("xfailif"); + marker_runtime_misuse(TESTING_MARKER_XFAILIF); } /// Return the host platform identifier used by collection-time marker probes. @@ -69,14 +78,14 @@ pub fn feature(_name: String) -> bool { /// /// `incan test` handles slow-test filtering. Calling this at runtime is a misuse. pub fn slow() { - marker_runtime_misuse("slow"); + marker_runtime_misuse(TESTING_MARKER_SLOW); } /// Marker runtime for `@std.testing.mark`. /// /// `incan test` handles marker selection during discovery. Calling this at runtime is a misuse. pub fn mark(_name: String) { - marker_runtime_misuse("mark"); + marker_runtime_misuse(TESTING_MARKER_MARK); } /// Marker runtime for `@std.testing.resource`. @@ -84,35 +93,35 @@ pub fn mark(_name: String) { /// `incan test` uses resource metadata to avoid overlapping generated test batches that declare the same resource. /// Calling this at runtime is a misuse. pub fn resource(_name: String) { - marker_runtime_misuse("resource"); + marker_runtime_misuse(TESTING_MARKER_RESOURCE); } /// Marker runtime for `@std.testing.serial`. /// /// `incan test` uses serial metadata to run a generated test batch alone. Calling this at runtime is a misuse. pub fn serial() { - marker_runtime_misuse("serial"); + marker_runtime_misuse(TESTING_MARKER_SERIAL); } /// Marker runtime for `@std.testing.timeout`. /// /// `incan test` uses timeout metadata when running generated test batches. Calling this at runtime is a misuse. pub fn timeout(_duration: String) { - marker_runtime_misuse("timeout"); + marker_runtime_misuse(TESTING_MARKER_TIMEOUT); } /// Marker runtime for `@std.testing.fixture`. /// /// `incan test` consumes fixture metadata during discovery. Calling this at runtime is a misuse. pub fn fixture() { - marker_runtime_misuse("fixture"); + marker_runtime_misuse(TESTING_MARKER_FIXTURE); } /// Marker runtime for `@std.testing.parametrize`. /// /// Parameter expansion is handled by `incan test`; calling this at runtime is a misuse. pub fn parametrize(_argnames: String, _argvalues: Vec) { - marker_runtime_misuse("parametrize"); + marker_runtime_misuse(TESTING_MARKER_PARAMETRIZE); } /// Parameter case wrapper for decorator metadata. @@ -185,7 +194,15 @@ mod tests { use std::any::Any; use std::panic; - use super::{fail_t, fixture, skip}; + use std::collections::HashSet; + + use super::{ + RUNNER_ONLY_MARKER_NAMES, TESTING_MARKER_FIXTURE, TESTING_MARKER_MARK, TESTING_MARKER_PARAMETRIZE, + TESTING_MARKER_RESOURCE, TESTING_MARKER_SERIAL, TESTING_MARKER_SKIP, TESTING_MARKER_SKIPIF, + TESTING_MARKER_SLOW, TESTING_MARKER_TEST, TESTING_MARKER_TIMEOUT, TESTING_MARKER_XFAIL, TESTING_MARKER_XFAILIF, + fail_t, fixture, mark, parametrize, resource, serial, skip, skipif, slow, test, + testing_marker_runtime_misuse_message, timeout, xfail, xfailif, + }; fn panic_message(payload: &(dyn Any + Send)) -> Option<&str> { if let Some(message) = payload.downcast_ref::() { @@ -195,48 +212,60 @@ mod tests { } } - #[test] - fn fail_t_panics_with_the_given_message() -> Result<(), Box> { - let result = panic::catch_unwind(|| fail_t::<()>("custom failure".to_string())); + fn assert_marker_runtime_misuse(marker: &str, call: F) -> Result<(), Box> + where + F: FnOnce() + panic::UnwindSafe, + { + let result = panic::catch_unwind(call); + let expected_message = testing_marker_runtime_misuse_message(marker); match result { - Ok(()) => Err(std::io::Error::other("fail_t returned instead of panicking").into()), + Ok(()) => Err(std::io::Error::other(format!("{marker} marker returned instead of panicking")).into()), Err(payload) => { - assert_eq!(panic_message(payload.as_ref()), Some("custom failure")); + assert_eq!(panic_message(payload.as_ref()), Some(expected_message.as_str())); Ok(()) } } } #[test] - fn marker_runtime_panics_explain_runner_only_usage() -> Result<(), Box> { - let result = panic::catch_unwind(|| skip("not implemented".to_string())); + fn fail_t_panics_with_the_given_message() -> Result<(), Box> { + let result = panic::catch_unwind(|| fail_t::<()>("custom failure".to_string())); match result { - Ok(()) => Err(std::io::Error::other("skip marker returned instead of panicking").into()), + Ok(()) => Err(std::io::Error::other("fail_t returned instead of panicking").into()), Err(payload) => { - assert_eq!( - panic_message(payload.as_ref()), - Some("std.testing.skip is marker metadata for `incan test` and is not executable runtime logic"), - ); + assert_eq!(panic_message(payload.as_ref()), Some("custom failure")); Ok(()) } } } #[test] - fn fixture_runtime_panics_explain_runner_only_usage() -> Result<(), Box> { - let result = panic::catch_unwind(fixture); + fn marker_runtime_panics_explain_runner_only_usage() -> Result<(), Box> { + assert_marker_runtime_misuse(TESTING_MARKER_TEST, test)?; + assert_marker_runtime_misuse(TESTING_MARKER_FIXTURE, fixture)?; + assert_marker_runtime_misuse(TESTING_MARKER_SKIP, || skip("not implemented".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_SKIPIF, || skipif(true, "not implemented".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_XFAIL, || xfail("known issue".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_XFAILIF, || xfailif(true, "known issue".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_SLOW, slow)?; + assert_marker_runtime_misuse(TESTING_MARKER_MARK, || mark("db".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_RESOURCE, || resource("db".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_SERIAL, serial)?; + assert_marker_runtime_misuse(TESTING_MARKER_TIMEOUT, || timeout("5s".to_string()))?; + assert_marker_runtime_misuse(TESTING_MARKER_PARAMETRIZE, || { + parametrize("value".to_string(), vec![1]); + })?; + Ok(()) + } - match result { - Ok(()) => Err(std::io::Error::other("fixture marker returned instead of panicking").into()), - Err(payload) => { - assert_eq!( - panic_message(payload.as_ref()), - Some("std.testing.fixture is marker metadata for `incan test` and is not executable runtime logic"), - ); - Ok(()) - } + #[test] + fn runner_only_marker_names_are_unique() { + let mut seen = HashSet::new(); + + for marker in RUNNER_ONLY_MARKER_NAMES { + assert!(seen.insert(marker), "duplicate std.testing marker name `{marker}`"); } } } diff --git a/crates/incan_stdlib/stdlib/compression/_auto.incn b/crates/incan_stdlib/stdlib/compression/_auto.incn index 5327b9c0f..b11e91add 100644 --- a/crates/incan_stdlib/stdlib/compression/_auto.incn +++ b/crates/incan_stdlib/stdlib/compression/_auto.incn @@ -10,11 +10,11 @@ the generic boundary can express owned reader adapters directly. """ from rust::std::io import Cursor, Read -from rust::bzip2::read @ "0.6" import BzDecoder -from rust::flate2::read @ "1" import GzDecoder, ZlibDecoder -from rust::snap::read @ "1" import FrameDecoder -from rust::xz2::read @ "0.1" import XzDecoder -from rust::zstd::stream::read @ "0.13" import Decoder as ZstdReadDecoder +from rust::bzip2::read import BzDecoder +from rust::flate2::read import GzDecoder, ZlibDecoder +from rust::snap::read import FrameDecoder +from rust::xz2::read import XzDecoder +from rust::zstd::stream::read import Decoder as ZstdReadDecoder from std.compression._core import ( Codec, CompressionError, diff --git a/crates/incan_stdlib/stdlib/compression/bz2.incn b/crates/incan_stdlib/stdlib/compression/bz2.incn index 70e3c41fb..c2abf3068 100644 --- a/crates/incan_stdlib/stdlib/compression/bz2.incn +++ b/crates/incan_stdlib/stdlib/compression/bz2.incn @@ -5,8 +5,8 @@ This module owns the byte-oriented bzip2 surface and translates portable levels """ from rust::std::io import Cursor, Read -from rust::bzip2 @ "0.6" import Compression as BzCompression -from rust::bzip2::read @ "0.6" import BzDecoder, BzEncoder +from rust::bzip2 import Compression as BzCompression +from rust::bzip2::read import BzDecoder, BzEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/deflate.incn b/crates/incan_stdlib/stdlib/compression/deflate.incn index 3701c7904..e9ad713b5 100644 --- a/crates/incan_stdlib/stdlib/compression/deflate.incn +++ b/crates/incan_stdlib/stdlib/compression/deflate.incn @@ -6,8 +6,8 @@ from autodetection. """ from rust::std::io import Cursor, Read -from rust::flate2 @ "1" import Compression as FlateCompression -from rust::flate2::read @ "1" import DeflateDecoder, DeflateEncoder +from rust::flate2 import Compression as FlateCompression +from rust::flate2::read import DeflateDecoder, DeflateEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/gzip.incn b/crates/incan_stdlib/stdlib/compression/gzip.incn index 15de2028c..25a7ea95f 100644 --- a/crates/incan_stdlib/stdlib/compression/gzip.incn +++ b/crates/incan_stdlib/stdlib/compression/gzip.incn @@ -6,8 +6,8 @@ Rust `flate2` reader adapters as the codec boundary. """ from rust::std::io import Cursor, Read -from rust::flate2 @ "1" import Compression as FlateCompression -from rust::flate2::read @ "1" import GzDecoder, GzEncoder +from rust::flate2 import Compression as FlateCompression +from rust::flate2::read import GzDecoder, GzEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/lzma.incn b/crates/incan_stdlib/stdlib/compression/lzma.incn index 565a390b0..b81ad7e13 100644 --- a/crates/incan_stdlib/stdlib/compression/lzma.incn +++ b/crates/incan_stdlib/stdlib/compression/lzma.incn @@ -5,7 +5,7 @@ The public `std.compression.lzma` name exposes XZ-framed LZMA-family data throug """ from rust::std::io import Cursor, Read -from rust::xz2::read @ "0.1" import XzDecoder, XzEncoder +from rust::xz2::read import XzDecoder, XzEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/snappy.incn b/crates/incan_stdlib/stdlib/compression/snappy.incn index a34b52f71..08c12895b 100644 --- a/crates/incan_stdlib/stdlib/compression/snappy.incn +++ b/crates/incan_stdlib/stdlib/compression/snappy.incn @@ -6,7 +6,7 @@ autodetection. Raw block helpers live under `std.compression.snappy.raw`. """ from rust::std::io import Cursor, Read -from rust::snap::read @ "1" import FrameDecoder, FrameEncoder +from rust::snap::read import FrameDecoder, FrameEncoder from std.compression._core import Codec, CompressionError, _codec_error, _reject_level, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/snappy/raw.incn b/crates/incan_stdlib/stdlib/compression/snappy/raw.incn index 879e6f7e5..a4c06c6d0 100644 --- a/crates/incan_stdlib/stdlib/compression/snappy/raw.incn +++ b/crates/incan_stdlib/stdlib/compression/snappy/raw.incn @@ -5,7 +5,7 @@ Raw Snappy is an advanced interop surface for systems that store individual Snap from `std.compression` autodetection because raw blocks have no stable stream signature. """ -from rust::snap::raw @ "1" import Decoder as RawDecoder, Encoder as RawEncoder +from rust::snap::raw import Decoder as RawDecoder, Encoder as RawEncoder from std.compression._core import Codec, CompressionError, _codec_error, _reject_level diff --git a/crates/incan_stdlib/stdlib/compression/zlib.incn b/crates/incan_stdlib/stdlib/compression/zlib.incn index 9940b91be..d0178f549 100644 --- a/crates/incan_stdlib/stdlib/compression/zlib.incn +++ b/crates/incan_stdlib/stdlib/compression/zlib.incn @@ -6,8 +6,8 @@ backend errors into `CompressionError`. """ from rust::std::io import Cursor, Read -from rust::flate2 @ "1" import Compression as FlateCompression -from rust::flate2::read @ "1" import ZlibDecoder, ZlibEncoder +from rust::flate2 import Compression as FlateCompression +from rust::flate2::read import ZlibDecoder, ZlibEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/compression/zstd.incn b/crates/incan_stdlib/stdlib/compression/zstd.incn index 44bdcc46a..08fa8a651 100644 --- a/crates/incan_stdlib/stdlib/compression/zstd.incn +++ b/crates/incan_stdlib/stdlib/compression/zstd.incn @@ -6,8 +6,8 @@ This module exposes zstd frames through one-shot byte helpers and keeps backend- """ from rust::std::io import Cursor, Read -from rust::zstd::stream @ "0.13" import decode_all, encode_all -from rust::zstd::stream::read @ "0.13" import Decoder as ZstdReadDecoder, Encoder as ZstdReadEncoder +from rust::zstd::stream import decode_all, encode_all +from rust::zstd::stream::read import Decoder as ZstdReadDecoder, Encoder as ZstdReadEncoder from std.compression._core import Codec, CompressionError, _codec_error, _level_or_default, _validate_chunk_size, _write_sink_bytes from std.fs import File from std.io import _BytesIO diff --git a/crates/incan_stdlib/stdlib/hash/_core.incn b/crates/incan_stdlib/stdlib/hash/_core.incn index 6e2eaf9f4..c720b0f63 100644 --- a/crates/incan_stdlib/stdlib/hash/_core.incn +++ b/crates/incan_stdlib/stdlib/hash/_core.incn @@ -6,16 +6,16 @@ This module owns the algorithm wrappers and value hashing paths. File and reader """ from rust::incan_stdlib::errors import raise_value_error -from rust::blake2 @ "0.10" import Blake2b512, Blake2s256 -from rust::blake3 @ "1" import Hasher as Blake3Hasher, hash as blake3_hash -from rust::md5 @ "0.10" import Md5 -from rust::sha1 @ "0.10" import Sha1 -from rust::sha2 @ "0.10" import Digest, Sha224, Sha256, Sha384, Sha512 -from rust::sha3 @ "0.10" import Sha3_224, Sha3_256, Sha3_384, Sha3_512, Shake128, Shake256 -from rust::sha3::digest @ "0.10" import ExtendableOutputReset, Update, XofReader -from rust::xxhash_rust::xxh3 @ "0.8" with ["xxh3"] import Xxh3Default, xxh3_64 as rust_xxh3_64, xxh3_128 as rust_xxh3_128 -from rust::xxhash_rust::xxh32 @ "0.8" with ["xxh32"] import Xxh32 -from rust::xxhash_rust::xxh64 @ "0.8" with ["xxh64"] import Xxh64 +from rust::blake2 import Blake2b512, Blake2s256 +from rust::blake3 import Hasher as Blake3Hasher, hash as blake3_hash +from rust::md5 import Md5 +from rust::sha1 import Sha1 +from rust::sha2 import Digest, Sha224, Sha256, Sha384, Sha512 +from rust::sha3 import Sha3_224, Sha3_256, Sha3_384, Sha3_512, Shake128, Shake256 +from rust::sha3::digest import ExtendableOutputReset, Update, XofReader +from rust::xxhash_rust::xxh3 with ["xxh3"] import Xxh3Default, xxh3_64 as rust_xxh3_64, xxh3_128 as rust_xxh3_128 +from rust::xxhash_rust::xxh32 with ["xxh32"] import Xxh32 +from rust::xxhash_rust::xxh64 with ["xxh64"] import Xxh64 from std.traits.error import Error diff --git a/crates/incan_syntax/src/ast/decls.rs b/crates/incan_syntax/src/ast/decls.rs index 1a94aa38d..335f0de74 100644 --- a/crates/incan_syntax/src/ast/decls.rs +++ b/crates/incan_syntax/src/ast/decls.rs @@ -391,6 +391,8 @@ pub enum ParamKind { pub struct Decorator { pub path: ImportPath, pub name: Ident, + /// Explicit call-site type arguments for decorator factories, as in `@factory[T](...)`. + pub type_args: Vec>, /// Whether the decorator was written with a call suffix, including zero-argument factory calls like `@factory()`. pub is_call: bool, pub args: Vec, diff --git a/crates/incan_syntax/src/ast/exprs.rs b/crates/incan_syntax/src/ast/exprs.rs index 95c26e49f..2788f22b0 100644 --- a/crates/incan_syntax/src/ast/exprs.rs +++ b/crates/incan_syntax/src/ast/exprs.rs @@ -182,10 +182,16 @@ pub struct ScopedSurfaceOwner { pub call: Option, } +#[derive(Debug, Clone, PartialEq)] +pub enum FStringFormat { + Display, + Debug, +} + #[derive(Debug, Clone, PartialEq)] pub enum FStringPart { Literal(String), - Expr(Spanned), + Expr { expr: Spanned, format: FStringFormat }, } /// Parsed integer literal with the **source substring** used for formatting. diff --git a/crates/incan_syntax/src/ast/imports.rs b/crates/incan_syntax/src/ast/imports.rs index b01c5efc8..558041c18 100644 --- a/crates/incan_syntax/src/ast/imports.rs +++ b/crates/incan_syntax/src/ast/imports.rs @@ -61,7 +61,7 @@ pub enum ImportKind { path: Vec, /// Optional version requirement string (Cargo semver syntax). version: Option, - /// Optional feature list (only valid when `version` is provided). + /// Optional feature list. features: Vec, }, /// `from rust::time import Instant, Duration` - Rust crate with specific items @@ -71,7 +71,7 @@ pub enum ImportKind { path: Vec, /// Optional version requirement string (Cargo semver syntax). version: Option, - /// Optional feature list (only valid when `version` is provided). + /// Optional feature list. features: Vec, items: Vec, }, diff --git a/crates/incan_syntax/src/diagnostics/catalog/errors/types.rs b/crates/incan_syntax/src/diagnostics/catalog/errors/types.rs index 2095cda06..3ba4ec969 100644 --- a/crates/incan_syntax/src/diagnostics/catalog/errors/types.rs +++ b/crates/incan_syntax/src/diagnostics/catalog/errors/types.rs @@ -186,6 +186,11 @@ pub fn decorator_factory_not_callable(path: &str, span: Span) -> CompileError { CompileError::type_error(format!("'{path}' does not return a callable"), span) } +/// Report a decorator callable whose application result is not callable. +pub fn decorator_result_not_callable(path: &str, span: Span) -> CompileError { + CompileError::type_error(format!("decorator '{path}' must return a callable"), span) +} + /// Report a type-valued decorator argument on a user-defined decorator factory. pub fn decorator_type_argument_not_supported(path: &str, span: Span) -> CompileError { CompileError::type_error( @@ -429,6 +434,15 @@ pub fn incompatible_error_type(expected: &str, found: &str, span: Span) -> Compi .with_hint("Use map_err to convert the error type, or add a From implementation") } +pub fn try_without_result_return(span: Span) -> CompileError { + CompileError::type_error( + "Cannot use '?' here: the enclosing function does not return Result[_, E]".to_string(), + span, + ) + .with_note("The '?' operator unwraps Ok(value) or returns early with Err(error)") + .with_hint("Change the enclosing function return type to Result[T, E], or handle the Result with match") +} + pub fn testing_marker_runtime_call_not_supported(name: &str, span: Span) -> CompileError { CompileError::type_error( format!("'{}' is a test marker decorator and cannot be called at runtime", name), diff --git a/crates/incan_syntax/src/parser/decl/decorators.rs b/crates/incan_syntax/src/parser/decl/decorators.rs index 617b979cb..104a6fa38 100644 --- a/crates/incan_syntax/src/parser/decl/decorators.rs +++ b/crates/incan_syntax/src/parser/decl/decorators.rs @@ -11,6 +11,7 @@ impl<'a> Parser<'a> { .last() .cloned() .ok_or_else(|| errors::decorator_path_expected(self.current_span()))?; + let type_args = self.call_site_type_args()?; let is_call = self.match_punct(PunctuationId::LParen); let args = if is_call { let args = self.decorator_args()?; @@ -24,6 +25,7 @@ impl<'a> Parser<'a> { Decorator { path, name, + type_args, is_call, args, }, diff --git a/crates/incan_syntax/src/parser/decl/imports.rs b/crates/incan_syntax/src/parser/decl/imports.rs index 284ad7247..694f61c38 100644 --- a/crates/incan_syntax/src/parser/decl/imports.rs +++ b/crates/incan_syntax/src/parser/decl/imports.rs @@ -159,8 +159,8 @@ impl<'a> Parser<'a> { if self.match_keyword(KeywordId::With) { features = self.string_list()?; } - } else if self.check_keyword(KeywordId::With) { - return Err(errors::rust_import_features_require_version(self.current_span())); + } else if self.match_keyword(KeywordId::With) { + features = self.string_list()?; } Ok((version, features)) diff --git a/crates/incan_syntax/src/parser/expr.rs b/crates/incan_syntax/src/parser/expr.rs index 22f0d2ba6..5bc0ea3c4 100644 --- a/crates/incan_syntax/src/parser/expr.rs +++ b/crates/incan_syntax/src/parser/expr.rs @@ -498,7 +498,7 @@ impl<'a> Parser<'a> { } /// Parse one call-site type argument: either a full [`Type`] or the inference placeholder `_`. - fn call_site_type_arg(&mut self) -> Result, CompileError> { + pub(super) fn call_site_type_arg(&mut self) -> Result, CompileError> { if let TokenKind::Ident(name) = &self.peek().kind && name == "_" { @@ -512,7 +512,7 @@ impl<'a> Parser<'a> { /// Parse optional explicit call-site type arguments (`[T, U]`) without consuming non-call brackets. /// /// This is intentionally conservative: we only treat brackets as call-site type args when the matching `]` is followed immediately by `(`. - fn call_site_type_args(&mut self) -> Result>, CompileError> { + pub(super) fn call_site_type_args(&mut self) -> Result>, CompileError> { if !self.check(&TokenKind::Punctuation(PunctuationId::LBracket)) { return Ok(Vec::new()); } @@ -1269,17 +1269,21 @@ impl<'a> Parser<'a> { } } + /// Convert lexer f-string segments into parsed AST parts while preserving interpolation spans and format markers. fn convert_fstring_parts(&self, parts: &[LexFStringPart]) -> Vec { parts .iter() .map(|p| match p { LexFStringPart::Literal(s) => FStringPart::Literal(s.clone()), LexFStringPart::Expr { text, offset } => { - // Parse simple field access chains like "user.name" or "obj.field.sub" let expr_span = Span::new(*offset, offset + text.len() + 2); - let mut expr = self.parse_fstring_expr(text); + let (expr_text, format) = split_fstring_format(text); + let mut expr = self.parse_fstring_expr(expr_text); self.shift_expr_spans(&mut expr, offset + 1); - FStringPart::Expr(Spanned::new(expr, expr_span)) + FStringPart::Expr { + expr: Spanned::new(expr, expr_span), + format, + } } }) .collect() @@ -1441,8 +1445,8 @@ impl<'a> Parser<'a> { } Expr::FString(parts) => { for part in parts { - if let FStringPart::Expr(value) = part { - self.shift_spanned_expr(value, offset); + if let FStringPart::Expr { expr, .. } = part { + self.shift_spanned_expr(expr, offset); } } } @@ -1547,7 +1551,6 @@ impl<'a> Parser<'a> { next_leading = 0; } } - self.expect(&TokenKind::Dedent, "Expected dedent after match body")?; let end = self.tokens[self.pos - 1].span.end; Ok(Spanned::new( @@ -2475,3 +2478,41 @@ impl<'a> Parser<'a> { } } + +/// Split a raw f-string interpolation body into expression text plus the supported top-level format marker. +fn split_fstring_format(text: &str) -> (&str, FStringFormat) { + let mut depth = 0usize; + let mut quote = None; + let mut escaped = false; + let mut format_colon = None; + + for (idx, ch) in text.char_indices() { + if let Some(active_quote) = quote { + if escaped { + escaped = false; + } else if ch == '\\' { + escaped = true; + } else if ch == active_quote { + quote = None; + } + continue; + } + + match ch { + '\'' | '"' => quote = Some(ch), + '(' | '[' | '{' => depth += 1, + ')' | ']' | '}' => depth = depth.saturating_sub(1), + ':' if depth == 0 => format_colon = Some(idx), + _ => {} + } + } + + if let Some(idx) = format_colon { + let spec = text[idx + 1..].trim(); + if spec == "?" { + return (text[..idx].trim_end(), FStringFormat::Debug); + } + } + + (text, FStringFormat::Display) +} diff --git a/crates/incan_syntax/src/parser/tests.rs b/crates/incan_syntax/src/parser/tests.rs index b795b115f..235c6397e 100644 --- a/crates/incan_syntax/src/parser/tests.rs +++ b/crates/incan_syntax/src/parser/tests.rs @@ -1249,6 +1249,28 @@ async def create() -> None: Ok(()) } + #[test] + fn test_parse_decorator_factory_with_explicit_type_args() -> Result<(), Vec> { + let source = r#" +@registered[(str) -> ColumnExpr]("inql.functions.col") +def col(name: str) -> ColumnExpr: + pass +"#; + let program = parse_str(source)?; + let func = match &program.declarations[0].node { + Declaration::Function(f) => f, + _ => panic!("Expected function"), + }; + let dec = &func.decorators[0].node; + assert_eq!(dec.path.segments, vec!["registered"]); + assert_eq!(dec.name, "registered"); + assert!(dec.is_call); + assert_eq!(dec.type_args.len(), 1); + assert!(matches!(&dec.type_args[0].node, Type::Function(_, _))); + assert_eq!(dec.args.len(), 1); + Ok(()) + } + #[test] fn test_parse_decorator_with_rust_namespace() -> Result<(), Vec> { // RFC 023: @rust.extern decorator must parse correctly (rust is a keyword) @@ -2223,16 +2245,26 @@ def identity( } #[test] - fn test_parse_rust_import_with_features_requires_version() { + fn test_parse_rust_import_with_features_without_inline_version() -> Result<(), Vec> { let source = r#"import rust::tokio with ["full"]"#; - let Err(err) = parse_str(source) else { - panic!("Expected rust import features to require version"); - }; - assert!( - err[0].message.contains("features require a version"), - "Unexpected error: {}", - err[0].message - ); + let program = parse_str(source)?; + match &program.declarations[0].node { + Declaration::Import(import) => match &import.kind { + ImportKind::RustCrate { + crate_name, + version, + features, + .. + } => { + assert_eq!(crate_name, "tokio"); + assert_eq!(version, &None); + assert_eq!(features, &vec!["full".to_string()]); + } + _ => panic!("Expected rust module import"), + }, + _ => panic!("Expected import"), + } + Ok(()) } #[test] @@ -3113,14 +3145,14 @@ def main() -> int: }; let first_expr = match &parts[1] { - FStringPart::Expr(expr) => expr, + FStringPart::Expr { expr, .. } => expr, _ => panic!("Expected first interpolation expression"), }; assert_eq!(first_expr.span.start, first_expected_start); assert_eq!(first_expr.span.end, first_expected_start + "{title}".len()); let second_expr = match &parts[3] { - FStringPart::Expr(expr) => expr, + FStringPart::Expr { expr, .. } => expr, _ => panic!("Expected second interpolation expression"), }; assert_eq!(second_expr.span.start, second_expected_start); @@ -3155,7 +3187,7 @@ def main() -> int: }; let interpolation = match &parts[1] { - FStringPart::Expr(expr) => expr, + FStringPart::Expr { expr, .. } => expr, _ => panic!("Expected interpolation expression"), }; @@ -3166,6 +3198,45 @@ def main() -> int: Ok(()) } + #[test] + fn test_parse_fstring_debug_format_marker() -> Result<(), Vec> { + let source = "def render(columns: list[str]) -> str:\n return f\"columns: {columns:?}\"\n"; + let program = parse_str(source)?; + + let function = match &program.declarations[0].node { + Declaration::Function(f) => f, + _ => panic!("Expected function"), + }; + + let return_expr = match &function.body[0].node { + Statement::Return(Some(expr)) => expr, + _ => panic!("Expected return with expression"), + }; + + let parts = match &return_expr.node { + Expr::FString(parts) => parts, + _ => panic!("Expected f-string expression"), + }; + + let expected_start = match source.find("{columns:?}") { + Some(start) => start, + None => panic!("Could not find interpolation in source"), + }; + let interpolation = match &parts[1] { + FStringPart::Expr { expr, format } => { + assert!(matches!(format, FStringFormat::Debug)); + expr + } + _ => panic!("Expected interpolation expression"), + }; + + assert_eq!(interpolation.span.start, expected_start); + assert_eq!(interpolation.span.end, expected_start + "{columns:?}".len()); + assert!(matches!(interpolation.node, Expr::Ident(ref name) if name == "columns")); + + Ok(()) + } + #[test] fn test_parse_fstring_expr_span_method_call_with_index() -> Result<(), Vec> { let source = "def render(users: List[str]) -> str:\n return f\"user: {users[unknown_idx].upper()}\"\n"; @@ -3192,7 +3263,7 @@ def main() -> int: }; let interpolation = match &parts[1] { - FStringPart::Expr(expr) => expr, + FStringPart::Expr { expr, .. } => expr, _ => panic!("Expected interpolation expression"), }; assert_eq!(interpolation.span.start, expected_start); @@ -3353,7 +3424,7 @@ def main() -> int: }; let interpolation = match &parts[1] { - FStringPart::Expr(expr) => expr, + FStringPart::Expr { expr, .. } => expr, _ => panic!("Expected interpolation expression"), }; assert_eq!(interpolation.span.start, expected_start); diff --git a/crates/rust_inspect/src/cache.rs b/crates/rust_inspect/src/cache.rs index f5880a8e0..b3c8954d0 100644 --- a/crates/rust_inspect/src/cache.rs +++ b/crates/rust_inspect/src/cache.rs @@ -91,7 +91,7 @@ struct DiskCacheEnvelope { } // Bump when extracted metadata semantics change in a way that makes previously persisted items unsafe to reuse. -const DISK_CACHE_FORMAT: u32 = 6; +const DISK_CACHE_FORMAT: u32 = 7; const DISK_CACHE_FILE: &str = ".incan_rust_inspect_cache.json"; // Backward-compatibility read path for caches written before the crate/module rename. const LEGACY_DISK_CACHE_FILE: &str = ".incan_rust_metadata_cache.json"; diff --git a/crates/rust_inspect/src/extractor.rs b/crates/rust_inspect/src/extractor.rs index 1bc194d11..30e43fa0c 100644 --- a/crates/rust_inspect/src/extractor.rs +++ b/crates/rust_inspect/src/extractor.rs @@ -5,7 +5,8 @@ use std::collections::BTreeMap; use incan_core::interop::{ RustFieldInfo, RustFunctionSig, RustImplementedTrait, RustItemKind, RustItemMetadata, RustMethodSig, RustModuleChild, RustModuleChildKind, RustModuleInfo, RustParam, RustTraitAssoc, RustTraitInfo, RustTypeInfo, - RustTypeShape, RustVariantInfo, RustVisibility, + RustTypeShape, RustVariantInfo, RustVisibility, render_rust_type_shape, split_top_level_rust_args, + strip_rust_borrow_lifetimes, }; use ra_ap_hir::{ Adt, AssocItem, Crate, DisplayTarget, Enum, FieldSource, Function, HasSource, HasVisibility, HirDisplay, Impl, @@ -116,33 +117,30 @@ fn canonical_adt_path(adt: Adt, db: &RootDatabase) -> Option { canonical_module_def_path(ModuleDef::Adt(adt), db) } -fn render_shape_display(shape: &RustTypeShape) -> String { - match shape { - RustTypeShape::Bool => "bool".to_string(), - RustTypeShape::Float => "f64".to_string(), - RustTypeShape::Int => "i64".to_string(), - RustTypeShape::Str => "String".to_string(), - RustTypeShape::Bytes => "Vec".to_string(), - RustTypeShape::Unit => "()".to_string(), - RustTypeShape::Option(inner) => format!("Option<{}>", render_shape_display(inner)), - RustTypeShape::Result(ok, err) => { - format!("Result<{}, {}>", render_shape_display(ok), render_shape_display(err)) - } - RustTypeShape::Tuple(items) => { - let rendered: Vec = items.iter().map(render_shape_display).collect(); - format!("({})", rendered.join(", ")) - } - RustTypeShape::Ref(inner) => format!("&{}", render_shape_display(inner)), - RustTypeShape::RustPath { path, args } => { - if args.is_empty() { - path.clone() - } else { - let rendered_args: Vec = args.iter().map(render_shape_display).collect(); - format!("{path}<{}>", rendered_args.join(", ")) - } - } - RustTypeShape::TypeParam(name) => name.clone(), - RustTypeShape::Unknown => "?".to_string(), +fn normalize_source_type_text(text: &str) -> String { + strip_rust_borrow_lifetimes(text).trim().replace(' ', "") +} + +fn borrowed_builtin_source_display(text: &str) -> Option { + let normalized = normalize_source_type_text(text); + let (prefix, inner) = if let Some(inner) = normalized.strip_prefix("&mut") { + ("&mut", inner) + } else if let Some(inner) = normalized.strip_prefix('&') { + ("&", inner) + } else { + return None; + }; + match inner { + "str" + | "[u8]" + | "String" + | "std::string::String" + | "alloc::string::String" + | "Vec" + | "std::vec::Vec" + | "alloc::vec::Vec" => Some(format!("{prefix}{inner}")), + _ if is_exact_numeric_display(inner) => Some(format!("{prefix}{inner}")), + _ => None, } } @@ -232,36 +230,8 @@ fn resolve_source_path(text: &str, crate_name: &str, module: Module, db: &RootDa None } -fn split_top_level_args(text: &str) -> Vec<&str> { - let mut args = Vec::new(); - let mut start = 0usize; - let mut angle = 0usize; - let mut paren = 0usize; - let mut bracket = 0usize; - for (idx, ch) in text.char_indices() { - match ch { - '<' => angle += 1, - '>' => angle = angle.saturating_sub(1), - '(' => paren += 1, - ')' => paren = paren.saturating_sub(1), - '[' => bracket += 1, - ']' => bracket = bracket.saturating_sub(1), - ',' if angle == 0 && paren == 0 && bracket == 0 => { - args.push(text[start..idx].trim()); - start = idx + 1; - } - _ => {} - } - } - let tail = text[start..].trim(); - if !tail.is_empty() { - args.push(tail); - } - args -} - fn source_type_shape(text: &str, crate_name: &str, module: Module, db: &RootDatabase) -> RustTypeShape { - let text = text.trim().replace(' ', ""); + let text = normalize_source_type_text(text); if text.is_empty() { return RustTypeShape::Unknown; } @@ -281,7 +251,7 @@ fn source_type_shape(text: &str, crate_name: &str, module: Module, db: &RootData return RustTypeShape::Ref(Box::new(source_type_shape(inner, crate_name, module, db))); } - if text == "[u8]" || text == "&[u8]" { + if text == "[u8]" { return RustTypeShape::Bytes; } @@ -291,7 +261,7 @@ fn source_type_shape(text: &str, crate_name: &str, module: Module, db: &RootData return RustTypeShape::Unit; } return RustTypeShape::Tuple( - split_top_level_args(inner) + split_top_level_rust_args(inner) .into_iter() .map(|arg| source_type_shape(arg, crate_name, module, db)) .collect(), @@ -304,7 +274,7 @@ fn source_type_shape(text: &str, crate_name: &str, module: Module, db: &RootData let base = resolve_source_path(&text[..start], crate_name, module, db).unwrap_or_else(|| text[..start].to_string()); let inner = &text[start + 1..text.len() - 1]; - let args: Vec = split_top_level_args(inner) + let args: Vec = split_top_level_rust_args(inner) .into_iter() .map(|arg| source_type_shape(arg, crate_name, module, db)) .collect(); @@ -456,7 +426,7 @@ fn function_sig_type_display(ty: &Type<'_>, db: &RootDatabase, dt: DisplayTarget } match rust_type_shape(ty, db, dt) { RustTypeShape::Unknown => raw, - other => render_shape_display(&other), + other => render_rust_type_shape(&other), } } @@ -469,6 +439,9 @@ fn function_sig_type_display(ty: &Type<'_>, db: &RootDatabase, dt: DisplayTarget fn source_function_return_type_display(f: Function, db: &RootDatabase) -> Option { let source = f.source(db)?; let text = source.value.ret_type()?.ty()?.to_string(); + if let Some(display) = borrowed_builtin_source_display(text.as_str()) { + return Some(display); + } let module = f.module(db); let crate_name = module .krate(db) @@ -480,7 +453,7 @@ fn source_function_return_type_display(f: Function, db: &RootDatabase) -> Option } Some(match shape { RustTypeShape::Unknown => normalize_display_path(text.as_str()), - other => render_shape_display(&other), + other => render_rust_type_shape(&other), }) } @@ -600,6 +573,9 @@ fn source_function_param_type_display(f: Function, param: &ra_ap_hir::Param<'_>, } let source_param = param_list.params().nth(param.index() - self_offset)?; let text = source_param.ty()?.to_string(); + if let Some(display) = borrowed_builtin_source_display(text.as_str()) { + return Some(display); + } if let Some(imported_display) = canonicalize_imported_single_segment_type_display(text.as_str(), f, db) { return Some(imported_display); } @@ -619,7 +595,7 @@ fn source_function_param_type_display(f: Function, param: &ra_ap_hir::Param<'_>, } let rendered = match shape { RustTypeShape::Unknown => normalize_display_path(text.as_str()), - other => render_shape_display(&other), + other => render_rust_type_shape(&other), }; if rendered.contains('?') && let Some(imported_display) = canonicalize_imported_single_segment_type_display(text.as_str(), f, db) @@ -636,7 +612,12 @@ fn extract_function_sig(f: Function, db: &RootDatabase, dt: DisplayTarget) -> Ru .map(|p| { let shape = rust_type_shape(p.ty(), db, dt); let mut type_display = function_sig_type_display(p.ty(), db, dt); - if (type_shape_contains_unknown(&shape) || p.ty().contains_unknown() || type_display.contains('?')) + if (type_shape_contains_unknown(&shape) + || p.ty().contains_unknown() + || type_display.contains('?') + || source_function_param_type_display(f, &p, db).is_some_and(|source_type_display| { + source_type_display.starts_with('&') && !type_display.starts_with('&') + })) && let Some(source_type_display) = source_function_param_type_display(f, &p, db) { type_display = source_type_display; @@ -648,8 +629,12 @@ fn extract_function_sig(f: Function, db: &RootDatabase, dt: DisplayTarget) -> Ru }) .collect(); let output_type = f.async_ret_type(db).unwrap_or_else(|| f.ret_type(db)); + let output_shape = rust_type_shape(&output_type, db, dt); let mut return_type = function_sig_type_display(&output_type, db, dt); - if return_type.starts_with("impl ") + if (return_type.starts_with("impl ") + || type_shape_contains_unknown(&output_shape) + || output_type.contains_unknown() + || return_type.contains('?')) && let Some(source_return_type) = source_function_return_type_display(f, db) { return_type = source_return_type; @@ -1074,4 +1059,57 @@ edition = "2021" assert_eq!(fields, ["zeta", "alpha"]); Ok(()) } + + #[test] + fn type_metadata_preserves_borrowed_slice_params_and_borrowed_option_returns() + -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + fs::create_dir_all(tmp.path().join("src"))?; + fs::write( + tmp.path().join("Cargo.toml"), + r#"[package] +name = "demo_borrow_probe" +version = "0.1.0" +edition = "2021" +"#, + )?; + fs::write( + tmp.path().join("src/lib.rs"), + r#"pub struct Codec; + +pub static CODEC: Codec = Codec; + +impl Codec { + pub fn for_label(label: &[u8]) -> Option<&'static Codec> { + let _ = label; + Some(&CODEC) + } + + pub fn decode<'a>(&'static self, bytes: &'a [u8]) -> (&'a [u8], &'static Codec, bool) { + (bytes, self, false) + } +} +"#, + )?; + + let workspace = RustWorkspace::load(tmp.path(), &|_| ())?; + let metadata = extract_rust_item(&workspace, "demo_borrow_probe::Codec")?; + let RustItemKind::Type(info) = metadata.kind else { + return Err(std::io::Error::other("expected type metadata").into()); + }; + let for_label = info + .methods + .iter() + .find(|method| method.name == "for_label") + .ok_or_else(|| std::io::Error::other("expected for_label metadata"))?; + assert_eq!(for_label.signature.params[0].type_display, "&[u8]"); + assert_eq!(for_label.signature.return_type, "Option<&demo_borrow_probe::Codec>"); + let decode = info + .methods + .iter() + .find(|method| method.name == "decode") + .ok_or_else(|| std::io::Error::other("expected decode metadata"))?; + assert_eq!(decode.signature.params[1].type_display, "&[u8]"); + Ok(()) + } } diff --git a/examples/pro/vocab_querykit/consumer/incan.lock b/examples/pro/vocab_querykit/consumer/incan.lock index 9dbbbe516..45532b2b7 100644 --- a/examples/pro/vocab_querykit/consumer/incan.lock +++ b/examples/pro/vocab_querykit/consumer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:d66866eca21aa7a29b265ef932049fe5b6da692cbe734cd4f7d300ce7163b359" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -50,7 +50,7 @@ dependencies = [ [[package]] name = "querykit_consumer" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", @@ -59,7 +59,7 @@ dependencies = [ [[package]] name = "querykit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/examples/pro/vocab_querykit/producer/incan.lock b/examples/pro/vocab_querykit/producer/incan.lock index 615fe6444..593a53823 100644 --- a/examples/pro/vocab_querykit/producer/incan.lock +++ b/examples/pro/vocab_querykit/producer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:17f122844d2fa1c9756f9a1976d222f15255557e74d975b8d8ff46536ea82b87" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -50,7 +50,7 @@ dependencies = [ [[package]] name = "querykit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/examples/pro/vocab_routekit/consumer/incan.lock b/examples/pro/vocab_routekit/consumer/incan.lock index 7e9a1589c..4b2181778 100644 --- a/examples/pro/vocab_routekit/consumer/incan.lock +++ b/examples/pro/vocab_routekit/consumer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:316bf142e6f8ea3b5838746eabec99c7e77d0acbcca01f8890c489b63498a743" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -59,7 +59,7 @@ dependencies = [ [[package]] name = "routekit_consumer" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", @@ -68,7 +68,7 @@ dependencies = [ [[package]] name = "routekit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/examples/pro/vocab_routekit/producer/incan.lock b/examples/pro/vocab_routekit/producer/incan.lock index 971820470..12b833d16 100644 --- a/examples/pro/vocab_routekit/producer/incan.lock +++ b/examples/pro/vocab_routekit/producer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:17f122844d2fa1c9756f9a1976d222f15255557e74d975b8d8ff46536ea82b87" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -59,7 +59,7 @@ dependencies = [ [[package]] name = "routekit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/examples/pro/vocab_studiokit/consumer/incan.lock b/examples/pro/vocab_studiokit/consumer/incan.lock index 04ee44ea8..0232e3728 100644 --- a/examples/pro/vocab_studiokit/consumer/incan.lock +++ b/examples/pro/vocab_studiokit/consumer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:e434303c58e58e0d05c2ffbd9b4c3b5a5984c4d74d64978e203d295f87495eae" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -89,7 +89,7 @@ dependencies = [ [[package]] name = "studiokit_consumer" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", @@ -98,7 +98,7 @@ dependencies = [ [[package]] name = "studiokit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/examples/pro/vocab_studiokit/producer/incan.lock b/examples/pro/vocab_studiokit/producer/incan.lock index 2ecb8140b..6fa7107b7 100644 --- a/examples/pro/vocab_studiokit/producer/incan.lock +++ b/examples/pro/vocab_studiokit/producer/incan.lock @@ -3,7 +3,7 @@ [incan] format = 1 -incan-version = "0.3.0-dev.51" +incan-version = "0.3.0-rc6" deps-fingerprint = "sha256:17f122844d2fa1c9756f9a1976d222f15255557e74d975b8d8ff46536ea82b87" cargo-features = [] cargo-no-default-features = false @@ -17,14 +17,14 @@ version = 4 [[package]] name = "incan_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "serde", ] [[package]] name = "incan_derive" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "proc-macro2", "quote", @@ -33,7 +33,7 @@ dependencies = [ [[package]] name = "incan_stdlib" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_core", "incan_derive", @@ -89,7 +89,7 @@ dependencies = [ [[package]] name = "studiokit_core" -version = "0.3.0-dev.51" +version = "0.3.0-rc6" dependencies = [ "incan_derive", "incan_stdlib", diff --git a/scripts/check_changed_rustdocs.py b/scripts/check_changed_rustdocs.py index 111fc9c4d..55ea0afc0 100644 --- a/scripts/check_changed_rustdocs.py +++ b/scripts/check_changed_rustdocs.py @@ -1,8 +1,8 @@ #!/usr/bin/env python3 -"""Fail when touched Rust source files contain undocumented non-test functions or methods. +"""Fail when changed Rust source files contain undocumented non-test functions or methods. -This script is intentionally scoped to changed `.rs` files so the branch enforces a boyscout-style documentation -standard without requiring an immediate repo-wide documentation migration. +By default, this checks both staged and unstaged `.rs` changes. Pass `--base ` or set `INCAN_RUSTDOC_GATE_BASE` +when a release or review branch needs to be checked against a comparison base such as `origin/release/v0.2`. Eventually, we can replace this script with the following clippy rules: #![warn(missing_docs)] @@ -11,6 +11,8 @@ from __future__ import annotations +import argparse +import os import re import subprocess import sys @@ -27,10 +29,16 @@ HUNK_RE = re.compile(r"^@@ -\d+(?:,\d+)? \+(?P\d+)(?:,(?P\d+))? @@") -def changed_rust_files() -> dict[Path, set[int]]: - """Return changed Rust source files and their changed current-file line numbers.""" +def merge_changed_lines(target: dict[Path, set[int]], source: dict[Path, set[int]]) -> None: + """Merge changed-line data from one parsed diff into `target`.""" + for path, lines in source.items(): + target.setdefault(path, set()).update(lines) + + +def changed_rust_files_from_diff_args(args: list[str]) -> dict[Path, set[int]]: + """Return changed Rust source files and current-file line numbers for one `git diff` invocation.""" result = subprocess.run( - ["git", "diff", "--unified=0", "--", "*.rs"], + args, cwd=ROOT, capture_output=True, text=True, @@ -64,7 +72,24 @@ def changed_rust_files() -> dict[Path, set[int]]: count = int(match.group("count") or "1") if count == 0: continue - files[current_path].update(range(start, start + count)) + files[current_path].update(range(start, start + count)) + return files + + +def changed_rust_files(base_ref: str | None) -> dict[Path, set[int]]: + """Return changed Rust source files and their changed current-file line numbers.""" + if base_ref: + return changed_rust_files_from_diff_args(["git", "diff", "--unified=0", base_ref, "--", "*.rs"]) + + files: dict[Path, set[int]] = {} + merge_changed_lines( + files, + changed_rust_files_from_diff_args(["git", "diff", "--unified=0", "--", "*.rs"]), + ) + merge_changed_lines( + files, + changed_rust_files_from_diff_args(["git", "diff", "--cached", "--unified=0", "--", "*.rs"]), + ) return files @@ -191,10 +216,22 @@ def missing_docs(path: Path, changed_lines: set[int]) -> list[tuple[int, str]]: return misses -def main() -> int: +def parse_args(argv: list[str]) -> argparse.Namespace: + """Parse command-line options for the rustdoc gate.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--base", + default=os.environ.get("INCAN_RUSTDOC_GATE_BASE"), + help="optional git ref to diff against instead of staged plus unstaged changes", + ) + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: """Run the touched-file rustdoc gate and print failures in `path:line:name` form.""" + args = parse_args(sys.argv[1:] if argv is None else argv) misses: list[tuple[Path, int, str]] = [] - for path, changed_lines in changed_rust_files().items(): + for path, changed_lines in changed_rust_files(args.base).items(): for line, name in missing_docs(path, changed_lines): misses.append((path, line, name)) diff --git a/src/backend/ir/codegen.rs b/src/backend/ir/codegen.rs index ccad9d95c..16efcea4b 100644 --- a/src/backend/ir/codegen.rs +++ b/src/backend/ir/codegen.rs @@ -33,719 +33,27 @@ use std::env; use std::path::PathBuf; use std::sync::Arc; -use crate::frontend::ast::{self, Declaration, Expr, ImportKind, ImportPath, Program}; -use crate::frontend::decorator_resolution; +use crate::frontend::ast::Program; use crate::frontend::diagnostics::CompileError; use crate::frontend::library_manifest_index::LibraryManifestIndex; -use crate::frontend::module::canonicalize_source_module_segments; -use crate::frontend::typechecker::stdlib_loader::StdlibAstCache; -use crate::library_manifest::{EnumValueExport, EnumValueTypeExport}; -use incan_core::lang::decorators::{self, DecoratorId}; -use incan_core::lang::traits::{self as core_traits, TraitId}; -use incan_core::lang::{stdlib, trait_capabilities}; - -use super::emit::{ExternalOrdinalCustomKey, ExternalOrdinalValueEnum}; + +use super::emit::CallableNameResolution; use super::scanners::{ check_for_this_import as scan_check_for_this_import, collect_rust_crates as scan_collect_rust_crates, detect_serde_usage, }; -use super::{AstLowering, EmitError, EmitService, IrEmitter, LoweringErrors}; - -const SERDE_SERIALIZE_DERIVE: &str = "serde::Serialize"; -const SERDE_DESERIALIZE_DERIVE: &str = "serde::Deserialize"; - -fn collect_model_field_aliases(main: &Program, deps: &[(&str, &Program)]) -> HashMap> { - use crate::frontend::ast::Declaration; - - let mut out: HashMap> = HashMap::new(); - - let mut visit = |p: &Program| { - for decl in &p.declarations { - let Declaration::Model(m) = &decl.node else { - continue; - }; - - let mut map: HashMap = HashMap::new(); - for f in &m.fields { - if let Some(alias) = &f.node.metadata.alias { - map.insert(alias.clone(), f.node.name.clone()); - } - } - - if !map.is_empty() { - out.entry(m.name.clone()).or_default().extend(map); - } - } - }; - - visit(main); - for (_, dep) in deps { - visit(dep); - } - - out -} - -/// Resolve a source import path to the generated Rust module path used for dependency emission. -fn generated_module_path_for_source_import(path: &ImportPath, current_module_path: &[String]) -> Option> { - let resolved_segments = if path.parent_levels > 0 { - let keep = current_module_path.len().checked_sub(path.parent_levels)?; - let mut resolved = current_module_path[..keep].to_vec(); - resolved.extend(path.segments.clone()); - resolved - } else { - path.segments.clone() - }; - let mut segments = canonicalize_source_module_segments(&resolved_segments); - - if segments.first().map(String::as_str) == Some(stdlib::STDLIB_ROOT) { - segments[0] = stdlib::INCAN_STD_NAMESPACE.to_string(); - } - - Some(segments) -} - -/// True when a dependency module should keep its public API even if the main module does not import every item. -fn should_preserve_dependency_public_items(module_path: &[String], preserve_non_stdlib_public_items: bool) -> bool { - if matches!( - module_path.first().map(String::as_str), - Some(stdlib::STDLIB_ROOT | stdlib::INCAN_STD_NAMESPACE) - ) { - return true; - } - preserve_non_stdlib_public_items -} - -/// Return whether a function carries the stdlib-backed web route decorator that lowers to a Rust proc-macro attribute. -/// -/// Binary-style dependency emission prunes otherwise-unreferenced private items. Route handlers are different because -/// their Rust attribute expands into inventory registration after IR emission, so the function itself is a generated -/// entrypoint even when no Incan expression calls it directly. -fn has_web_route_passthrough_decorator( - func: &ast::FunctionDecl, - aliases: &HashMap>, - stdlib_cache: &mut StdlibAstCache, -) -> bool { - func.decorators.iter().any(|decorator| { - let resolved = decorator_resolution::resolve_decorator_path(&decorator.node, aliases); - if resolved.len() < 2 { - return false; - } - let module_segments = &resolved[..resolved.len() - 1]; - let name = &resolved[resolved.len() - 1]; - if name != "route" { - return false; - } - let Some(meta) = stdlib_cache.lookup_function_meta(module_segments, name) else { - return false; - }; - meta.is_rust_extern && meta.rust_module_path.as_deref() == Some("incan_web_macros") - }) -} - -/// Collect dependency-module declarations that are referenced through imports. -fn collect_externally_reachable_items_by_module( - main: &Program, - dependency_modules: &[(&str, &Program, Option>)], -) -> HashMap, HashSet> { - let module_paths: HashSet> = dependency_modules - .iter() - .map(|(name, _, path_segments)| path_segments.clone().unwrap_or_else(|| vec![(*name).to_string()])) - .collect(); - - /// Record imported item names against the generated dependency module that owns them. - fn record_imports( - reachable: &mut HashMap, HashSet>, - program: &Program, - current_module_path: &[String], - module_paths: &HashSet>, - ) { - if crate::frontend::surface_semantics::uses_ambient_log_surface(program) { - reachable - .entry(vec!["std".to_string(), "logging".to_string()]) - .or_default() - .insert("get_logger".to_string()); - } - let mut module_import_bindings: HashMap> = HashMap::new(); - for decl in &program.declarations { - let Declaration::Import(import) = &decl.node else { - continue; - }; - match &import.kind { - ImportKind::From { module, items } => { - let Some(module_path) = generated_module_path_for_source_import(module, current_module_path) else { - continue; - }; - let reachable_items = reachable.entry(module_path).or_default(); - for item in items { - reachable_items.insert(item.name.clone()); - } - } - ImportKind::Module(path) => { - let Some(segments) = generated_module_path_for_source_import(path, current_module_path) else { - continue; - }; - if module_paths.contains(&segments) { - if let Some(binding) = import.alias.clone().or_else(|| path.segments.last().cloned()) { - module_import_bindings.insert(binding, segments); - } - continue; - } - let Some(item_name) = segments.last() else { - continue; - }; - for module_path in module_paths { - if segments.len() == module_path.len() + 1 && segments.starts_with(module_path) { - reachable - .entry(module_path.clone()) - .or_default() - .insert(item_name.clone()); - break; - } - } - } - ImportKind::PubLibrary { .. } - | ImportKind::PubFrom { .. } - | ImportKind::RustCrate { .. } - | ImportKind::RustFrom { .. } - | ImportKind::Python(_) => {} - } - } - if !module_import_bindings.is_empty() { - let _ = crate::frontend::ast_walk::any_expr_in_program(program, |expr| { - if let Expr::Field(object, field) = expr - && let Expr::Ident(binding) = &object.node - && let Some(module_path) = module_import_bindings.get(binding) - { - reachable.entry(module_path.clone()).or_default().insert(field.clone()); - } - if let Expr::MethodCall(object, method, _, _) = expr - && let Expr::Ident(binding) = &object.node - && let Some(module_path) = module_import_bindings.get(binding) - { - reachable.entry(module_path.clone()).or_default().insert(method.clone()); - } - false - }); - } - if module_paths.contains(current_module_path) { - let aliases = decorator_resolution::collect_import_aliases(program); - let mut stdlib_cache = StdlibAstCache::new(); - for decl in &program.declarations { - let Declaration::Function(func) = &decl.node else { - continue; - }; - if has_web_route_passthrough_decorator(func, &aliases, &mut stdlib_cache) { - reachable - .entry(current_module_path.to_vec()) - .or_default() - .insert(func.name.clone()); - } - } - } - } - - let mut reachable = HashMap::new(); - record_imports(&mut reachable, main, &[String::from("main")], &module_paths); - for (name, program, path_segments) in dependency_modules { - let module_path = path_segments.clone().unwrap_or_else(|| vec![(*name).to_string()]); - record_imports(&mut reachable, program, &module_path, &module_paths); - } - reachable -} - -/// Dependency type facts gathered during codegen setup and reused by module emission. -/// -/// Multi-file consumers only carry short nominal type names after typechecking/lowering, so emission cannot infer -/// imported-enum ownership rules from local IR alone. This metadata keeps a single codegen-owned source of truth for: -/// - dependency module qualification (`module_paths`) -/// - short-name collisions that must not be auto-qualified (`ambiguous_type_names`) -/// - imported enum names that are safe to treat as enum loop elements (`enum_type_names`) -/// - imported stdlib error types whose trait methods require Rust trait imports (`error_trait_type_names`) -#[derive(Debug, Clone, Default)] -struct DependencyTypeMetadata { - module_paths: HashMap>, - ambiguous_type_names: HashSet, - enum_type_names: HashSet, - error_trait_type_names: HashSet, -} +use super::{AstLowering, EmitError, EmitService, FunctionRegistry, IrEmitter, IrProgram, LoweringErrors}; -/// Collect dependency type metadata needed by IR emission for cross-module nominal types. -/// -/// Enum loop ownership is the subtle case: imported enums lower to nominal `Struct(name)` references in consumer -/// modules, so the emitter cannot rely on local enum declarations when deciding whether `list[T]` loops should emit -/// `.iter().cloned()`. This helper records enum names from dependency modules while excluding ambiguous short names and -/// short names that are also used by non-enum dependency types. -fn collect_dependency_type_metadata(deps: &[(&str, &Program, Option>)]) -> DependencyTypeMetadata { - let mut paths: HashMap> = HashMap::new(); - let mut ambiguous: HashSet = HashSet::new(); - let mut enum_type_names: HashSet = HashSet::new(); - let mut non_enum_type_names: HashSet = HashSet::new(); - let mut error_trait_type_names: HashSet = HashSet::new(); - let error_trait_name = core_traits::as_str(TraitId::Error); - - for (_name, program, path_segments) in deps { - for decl in &program.declarations { - let type_name = match &decl.node { - Declaration::Model(m) => { - if m.traits.iter().any(|bound| bound.node.name == error_trait_name) { - error_trait_type_names.insert(m.name.clone()); - } - Some((&m.name, false)) - } - Declaration::Class(c) => { - if c.traits.iter().any(|bound| bound.node.name == error_trait_name) { - error_trait_type_names.insert(c.name.clone()); - } - Some((&c.name, false)) - } - Declaration::Enum(e) => Some((&e.name, true)), - Declaration::TypeAlias(a) => Some((&a.name, false)), - Declaration::Newtype(n) => Some((&n.name, false)), - _ => None, - }; - let Some((name, is_enum)) = type_name else { - continue; - }; +mod dependency_metadata; +mod ordinal_bridge; +mod serde_activation; - if is_enum { - enum_type_names.insert(name.clone()); - } else { - non_enum_type_names.insert(name.clone()); - } - - let Some(segs) = path_segments.as_ref() else { - continue; - }; - - if let Some(existing) = paths.get(name) { - if existing != segs { - ambiguous.insert(name.clone()); - } - } else { - paths.insert(name.clone(), segs.clone()); - } - } - } - - for name in &ambiguous { - paths.remove(name); - } - enum_type_names.retain(|name| !ambiguous.contains(name) && !non_enum_type_names.contains(name)); - - DependencyTypeMetadata { - module_paths: paths, - ambiguous_type_names: ambiguous, - enum_type_names, - error_trait_type_names, - } -} - -/// Return whether a program imports the stdlib ordinal-map contract. -fn imports_std_ordinal_contract(program: &Program) -> bool { - let capability = trait_capabilities::stable_ordinal_key(); - program.declarations.iter().any(|decl| { - let Declaration::Import(import) = &decl.node else { - return false; - }; - match &import.kind { - ImportKind::Module(_) => false, - ImportKind::From { module, items } if import_path_matches_capability(module, capability) => items - .iter() - .any(|item| trait_capabilities::import_triggers_capability(capability, item.name.as_str())), - _ => false, - } - }) -} - -/// Return whether an import path names the module that owns a temporary capability contract. -fn import_path_matches_capability(path: &ImportPath, capability: &trait_capabilities::TraitCapabilityInfo) -> bool { - trait_capabilities::module_path_matches(capability, &path.segments) -} - -/// Return whether any module in the current compilation needs value-enum `OrdinalKey` impls. -fn compilation_imports_std_ordinal_contract(main: &Program, deps: &[(&str, &Program, Option>)]) -> bool { - imports_std_ordinal_contract(main) || deps.iter().any(|(_, program, _)| imports_std_ordinal_contract(program)) -} - -/// Collect public scalar value enums from loaded `.incnlib` dependencies. -fn external_ordinal_value_enums(index: Option<&Arc>) -> Vec { - let Some(index) = index else { - return Vec::new(); - }; - let mut out = Vec::new(); - for dependency_key in index.known_libraries() { - let Some(crate::frontend::library_manifest_index::LibraryManifestIndexEntry::Loaded { manifest, metadata }) = - index.get(&dependency_key) - else { - continue; - }; - for enum_export in &manifest.exports.enums { - let Some(value_type) = enum_export.value_type else { - continue; - }; - let value_type = match value_type { - EnumValueTypeExport::Str => super::decl::IrEnumValueType::String, - EnumValueTypeExport::Int => super::decl::IrEnumValueType::Int, - }; - let mut values = Vec::new(); - let mut complete = true; - for variant in &enum_export.variants { - let Some(value) = &variant.value else { - complete = false; - break; - }; - values.push(match value { - EnumValueExport::Str(value) => super::decl::IrEnumValue::String(value.clone()), - EnumValueExport::Int(value) => super::decl::IrEnumValue::Int(*value), - }); - } - if !complete { - continue; - } - out.push(ExternalOrdinalValueEnum { - dependency_key: dependency_key.clone(), - name: enum_export.name.clone(), - type_identity: enum_export - .ordinal_type_identity - .clone() - .unwrap_or_else(|| format!("{}.{}", metadata.manifest_name, enum_export.name)), - value_type, - values, - }); - } - } - out -} - -/// Return whether a serialized trait bound names the std `OrdinalKey` capability. -fn type_bound_matches_ordinal_key(bound: &crate::library_manifest::TypeBoundExport) -> bool { - let capability = trait_capabilities::stable_ordinal_key(); - let trait_name = bound - .source_name - .as_deref() - .unwrap_or_else(|| bound.name.rsplit('.').next().unwrap_or(bound.name.as_str())); - if trait_name != capability.trait_name { - return false; - } - let Some(module_path) = &bound.module_path else { - return false; - }; - trait_capabilities::module_path_matches(capability, module_path) -} - -/// Return whether any exported trait adoption satisfies the std `OrdinalKey` contract. -fn export_adopts_ordinal_key( - trait_adoptions: &[crate::library_manifest::TypeBoundExport], - traits: &HashMap, -) -> bool { - trait_adoptions - .iter() - .any(|bound| type_bound_matches_ordinal_key(bound) || trait_bound_extends_ordinal_key(bound, traits)) -} - -/// Return whether a serialized trait bound resolves transitively to std `OrdinalKey`. -fn trait_bound_extends_ordinal_key( - bound: &crate::library_manifest::TypeBoundExport, - traits: &HashMap, -) -> bool { - let mut seen = HashSet::new(); - let mut work = vec![bound.name.as_str()]; - while let Some(name) = work.pop() { - if !seen.insert(name.to_string()) { - continue; - } - let Some(trait_export) = traits.get(name) else { - continue; - }; - for supertrait in &trait_export.supertraits { - if type_bound_matches_ordinal_key(supertrait) { - return true; - } - work.push(supertrait.name.as_str()); - } - } - false -} - -/// Return lookup keys for a manifest trait export, including its original source name when reexported under an alias. -fn trait_export_lookup_keys(trait_export: &crate::library_manifest::TraitExport) -> Vec { - let mut keys = vec![trait_export.name.clone()]; - if let Some(source_name) = &trait_export.source_name - && source_name != &trait_export.name - { - keys.push(source_name.clone()); - } - keys -} - -/// Return whether a manifest method set exposes a source method or its generated alias. -fn export_methods_include(methods: &[crate::library_manifest::MethodExport], name: &str) -> bool { - methods - .iter() - .any(|method| method.name == name || method.alias_of.as_deref() == Some(name)) -} - -/// Build custom-key bridge metadata for one exported concrete type when it adopts `OrdinalKey`. -fn external_ordinal_custom_key( - dependency_key: &str, - name: &str, - type_params: &[crate::library_manifest::TypeParamExport], - trait_adoptions: &[crate::library_manifest::TypeBoundExport], - methods: &[crate::library_manifest::MethodExport], - traits: &HashMap, -) -> Option { - if !type_params.is_empty() || !export_adopts_ordinal_key(trait_adoptions, traits) { - return None; - } - let hooks = trait_capabilities::stable_ordinal_key().bridge_hooks?; - Some(ExternalOrdinalCustomKey { - dependency_key: dependency_key.to_string(), - name: name.to_string(), - has_ordinal_hash: export_methods_include(methods, hooks.hash_method), - has_ordinal_bytes_equal: export_methods_include(methods, hooks.bytes_equal_method), - }) -} - -/// Collect public user-authored `OrdinalKey` adopters from loaded `.incnlib` dependencies. -fn external_ordinal_custom_keys(index: Option<&Arc>) -> Vec { - let Some(index) = index else { - return Vec::new(); - }; - let mut out = Vec::new(); - for dependency_key in index.known_libraries() { - let Some(crate::frontend::library_manifest_index::LibraryManifestIndexEntry::Loaded { manifest, .. }) = - index.get(&dependency_key) - else { - continue; - }; - let traits = manifest - .exports - .traits - .iter() - .flat_map(|trait_export| { - trait_export_lookup_keys(trait_export) - .into_iter() - .map(move |key| (key, trait_export)) - }) - .collect::>(); - for model in &manifest.exports.models { - if let Some(key) = external_ordinal_custom_key( - &dependency_key, - &model.name, - &model.type_params, - &model.trait_adoptions, - &model.methods, - &traits, - ) { - out.push(key); - } - } - for class in &manifest.exports.classes { - if let Some(key) = external_ordinal_custom_key( - &dependency_key, - &class.name, - &class.type_params, - &class.trait_adoptions, - &class.methods, - &traits, - ) { - out.push(key); - } - } - for newtype in &manifest.exports.newtypes { - if let Some(key) = external_ordinal_custom_key( - &dependency_key, - &newtype.name, - &newtype.type_params, - &newtype.trait_adoptions, - &newtype.methods, - &traits, - ) { - out.push(key); - } - } - for enum_export in &manifest.exports.enums { - if enum_export.value_type.is_some() { - continue; - } - if let Some(key) = external_ordinal_custom_key( - &dependency_key, - &enum_export.name, - &enum_export.type_params, - &enum_export.trait_adoptions, - &enum_export.methods, - &traits, - ) { - out.push(key); - } - } - } - out -} - -#[derive(Debug, Clone)] -struct OrdinalBridgeConfig { - emit_std_ordinal_value_enum_impls: bool, - external_value_enums: Vec, - external_custom_keys: Vec, -} - -impl OrdinalBridgeConfig { - /// Build a bridge configuration for generated internal modules. - fn for_internal_module(uses_std_ordinal_contract: bool) -> Self { - Self { - emit_std_ordinal_value_enum_impls: uses_std_ordinal_contract, - external_value_enums: Vec::new(), - external_custom_keys: Vec::new(), - } - } - - /// Build a bridge configuration for crate-root emission where dependency adapters live. - fn for_crate_root(uses_std_ordinal_contract: bool, index: Option<&Arc>) -> Self { - if !uses_std_ordinal_contract { - return Self::for_internal_module(false); - } - Self { - emit_std_ordinal_value_enum_impls: true, - external_value_enums: external_ordinal_value_enums(index), - external_custom_keys: external_ordinal_custom_keys(index), - } - } -} - -/// Return whether any loaded module derives serde serialize or deserialize through resolved JSON derive imports. -fn collect_serde_derives(main: &Program, deps: &[(&str, &Program)]) -> (bool, bool) { - let mut has_serialize = false; - let mut has_deserialize = false; - - let mut visit = |program: &Program| { - let import_aliases = decorator_resolution::collect_import_aliases(program); - for decl in &program.declarations { - let decorators = match &decl.node { - Declaration::Model(m) => Some(&m.decorators), - Declaration::Class(c) => Some(&c.decorators), - Declaration::Enum(e) => Some(&e.decorators), - _ => None, - }; - let Some(decorators) = decorators else { - continue; - }; - for dec in decorators { - if decorators::from_str(dec.node.name.as_str()) != Some(DecoratorId::Derive) { - continue; - } - for arg in &dec.node.args { - let crate::frontend::ast::DecoratorArg::Positional(expr) = arg else { - continue; - }; - let crate::frontend::ast::Expr::Ident(name) = &expr.node else { - continue; - }; - let resolved = import_aliases - .get(name) - .cloned() - .unwrap_or_else(|| vec![name.to_string()]); - match resolved.as_slice() { - [std, serde, json] if std == "std" && serde == "serde" && json == "json" => { - has_serialize = true; - has_deserialize = true; - } - [std, serde, json, trait_name] - if std == "std" && serde == "serde" && json == "json" && trait_name == "Serialize" => - { - has_serialize = true; - } - [std, serde, json, trait_name] - if std == "std" && serde == "serde" && json == "json" && trait_name == "Deserialize" => - { - has_deserialize = true; - } - [serde, trait_name] if serde == "serde" && trait_name == "Serialize" => { - has_serialize = true; - } - [serde, trait_name] if serde == "serde" && trait_name == "Deserialize" => { - has_deserialize = true; - } - _ => {} - } - } - } - } - }; - - visit(main); - for (_, dep) in deps { - visit(dep); - } - - // Fallback: if no explicit serde derive was found but serde usage is detected (e.g. `json_stringify()` builtin), we - // conservatively enable Serialize only. - // Deserialize is NOT enabled here because implicit serde usage (like `json_stringify`) - // only needs serialization, not deserialization. - if !has_serialize && !has_deserialize { - let serde_used = super::scanners::detect_serde_usage(main) - || deps - .iter() - .any(|(_, program)| super::scanners::detect_serde_usage(program)); - if serde_used { - has_serialize = true; - } - } - - (has_serialize, has_deserialize) -} - -/// Add serde derives to generated newtypes when the current program needs serde support. -fn add_serde_to_newtypes(ir_program: &mut super::IrProgram, add_serialize: bool, add_deserialize: bool) { - use super::decl::IrDeclKind; - use super::types::IrType; - - /// Return whether a newtype inner type can safely receive derived serde support. - fn is_conservative_serde_safe_newtype_inner(ty: &IrType) -> bool { - match ty { - IrType::Unit - | IrType::Bool - | IrType::Int - | IrType::Float - | IrType::String - | IrType::Bytes - | IrType::StaticStr - | IrType::StaticBytes - | IrType::FrozenStr - | IrType::FrozenBytes - | IrType::StrRef => true, - IrType::List(inner) | IrType::Set(inner) | IrType::Option(inner) => { - is_conservative_serde_safe_newtype_inner(inner) - } - IrType::Dict(key, value) | IrType::Result(key, value) => { - is_conservative_serde_safe_newtype_inner(key) && is_conservative_serde_safe_newtype_inner(value) - } - IrType::Tuple(items) => items.iter().all(is_conservative_serde_safe_newtype_inner), - _ => false, - } - } - - for decl in &mut ir_program.declarations { - if let IrDeclKind::Struct(s) = &mut decl.kind - && s.fields.len() == 1 - && s.fields[0].name == "0" - { - if !s.type_params.is_empty() { - continue; - } - if !is_conservative_serde_safe_newtype_inner(&s.fields[0].ty) { - continue; - } - if add_serialize && !s.derives.iter().any(|d| d == SERDE_SERIALIZE_DERIVE) { - s.derives.push(SERDE_SERIALIZE_DERIVE.to_string()); - } - if add_deserialize && !s.derives.iter().any(|d| d == SERDE_DESERIALIZE_DERIVE) { - s.derives.push(SERDE_DESERIALIZE_DERIVE.to_string()); - } - } - } -} +use dependency_metadata::{ + DependencySymbolMetadata, collect_dependency_symbol_metadata, collect_externally_reachable_items_by_module, + collect_model_field_aliases, should_preserve_dependency_public_items, +}; +use ordinal_bridge::{OrdinalBridgeConfig, compilation_imports_std_ordinal_contract, imports_std_ordinal_contract}; +use serde_activation::{add_serde_to_newtypes, collect_serde_derives}; /// Error during Rust code generation. /// @@ -858,6 +166,8 @@ pub struct IrCodegen<'a> { strict_generated_lints: bool, /// Private IR items called by generated code that is appended outside normal IR emission. externally_reachable_items: HashSet, + /// Private dependency-module IR items called by generated code appended inside that module. + externally_reachable_items_by_module: HashMap, HashSet>, /// Public serialized value-enum identities for library builds, keyed by source identity (`module.Type`). public_ordinal_type_identities: HashMap, /// Whether non-stdlib dependency modules keep public items that are not otherwise reachable. @@ -882,6 +192,7 @@ impl<'a> IrCodegen<'a> { library_manifest_index: None, strict_generated_lints: false, externally_reachable_items: HashSet::new(), + externally_reachable_items_by_module: HashMap::new(), public_ordinal_type_identities: HashMap::new(), preserve_dependency_public_items: true, #[cfg(feature = "rust_inspect")] @@ -889,6 +200,69 @@ impl<'a> IrCodegen<'a> { } } + /// Build a registry for explicit canonical cross-module calls. + fn canonical_registry_for_programs<'program>( + programs: impl IntoIterator, + ) -> FunctionRegistry { + let programs: Vec<_> = programs.into_iter().collect(); + let mut registry = FunctionRegistry::new(); + for (module_path, program) in &programs { + for (name, signature) in program.function_registry.iter() { + let mut canonical_path = (*module_path).to_vec(); + canonical_path.push(name.clone()); + registry.register_canonical_path( + &canonical_path, + signature.params.clone(), + signature.return_type.clone(), + ); + } + } + + let mut pending_reexports = Vec::new(); + for (module_path, program) in &programs { + for reexport in &program.function_reexports { + let mut alias_path = (*module_path).to_vec(); + alias_path.push(reexport.name.clone()); + pending_reexports.push((alias_path, reexport.target_path.clone())); + } + } + while !pending_reexports.is_empty() { + let mut unresolved = Vec::new(); + let mut made_progress = false; + for (alias_path, target_path) in pending_reexports { + if registry.get_canonical_path(&alias_path).is_some() { + made_progress = true; + continue; + } + if let Some(signature) = registry.get_canonical_path(&target_path).cloned() { + registry.register_canonical_path( + &alias_path, + signature.params.clone(), + signature.return_type.clone(), + ); + made_progress = true; + } else { + unresolved.push((alias_path, target_path)); + } + } + if !made_progress { + break; + } + pending_reexports = unresolved; + } + registry + } + + fn apply_dependency_symbol_metadata(emitter: &mut IrEmitter<'_>, metadata: &DependencySymbolMetadata) { + emitter.set_type_module_paths(metadata.module_paths.clone(), metadata.ambiguous_type_names.clone()); + emitter.set_value_module_paths( + metadata.value_module_paths.clone(), + metadata.ambiguous_value_names.clone(), + ); + emitter.set_dependency_enum_types(metadata.enum_type_names.clone()); + emitter.set_external_error_trait_types(metadata.error_trait_type_names.clone()); + } + /// Enable strict generated Rust lint validation for `--emit-rust --strict`. pub fn set_strict_generated_lints(&mut self, enabled: bool) { self.strict_generated_lints = enabled; @@ -899,6 +273,11 @@ impl<'a> IrCodegen<'a> { self.externally_reachable_items = names; } + /// Set private generated Rust entrypoints called by code injected into dependency modules. + pub fn set_externally_reachable_items_by_module(&mut self, names: HashMap, HashSet>) { + self.externally_reachable_items_by_module = names; + } + /// Set public serialized value-enum identities for library emission. pub fn set_public_ordinal_type_identities(&mut self, identities: HashMap) { self.public_ordinal_type_identities = identities; @@ -1144,7 +523,7 @@ impl<'a> IrCodegen<'a> { program: &Program, internal_module_roots: &HashSet, ) -> Result { - self.try_generate_via_ir_with_union_config(program, internal_module_roots, HashMap::new(), false) + self.try_generate_via_ir_with_union_config(program, internal_module_roots, HashMap::new(), false, None, None) } /// Generate code via the IR pipeline with optional crate-root union sharing for multi-file source modules. @@ -1154,6 +533,8 @@ impl<'a> IrCodegen<'a> { internal_module_roots: &HashSet, generated_union_types: HashMap, qualify_union_types_from_crate: bool, + mut callable_name_resolutions: Option<&mut HashMap>, + mut callable_name_used_signature_keys: Option<&mut HashSet>, ) -> Result { let deps: Vec<(&str, &Program)> = self .dependency_modules @@ -1164,7 +545,7 @@ impl<'a> IrCodegen<'a> { // RFC 021: Make alias-aware lowering work across module boundaries by seeding alias maps // for models declared in dependency modules as well. let global_aliases = collect_model_field_aliases(program, &deps); - let dependency_type_metadata = collect_dependency_type_metadata(&self.dependency_modules); + let dependency_symbol_metadata = collect_dependency_symbol_metadata(&self.dependency_modules); let uses_std_ordinal_contract = compilation_imports_std_ordinal_contract(program, &self.dependency_modules); let ordinal_bridge = self.ordinal_bridge_config(uses_std_ordinal_contract); let (needs_serialize, needs_deserialize) = collect_serde_derives(program, &deps); @@ -1199,25 +580,69 @@ impl<'a> IrCodegen<'a> { // RFC 023: Infer trait bounds for generic functions. super::trait_bound_inference::infer_trait_bounds(&mut ir_program); + let callable_name_use_facts = IrEmitter::callable_name_use_facts_for_program( + &ir_program, + &self.externally_reachable_items, + true, + &dependency_symbol_metadata.error_trait_type_names, + ); + if let Some(used_keys) = callable_name_used_signature_keys.as_deref_mut() { + used_keys.extend(callable_name_use_facts.signature_keys.iter().cloned()); + if callable_name_use_facts.generic_trait_used { + used_keys.extend(callable_name_use_facts.function_arg_signature_keys.iter().cloned()); + } + } + if let Some(resolutions) = callable_name_resolutions.as_deref_mut() { + IrEmitter::add_callable_name_resolutions_for_program(resolutions, Vec::new(), &ir_program); + } + let callable_name_resolutions_for_emit = callable_name_resolutions + .as_ref() + .map(|resolutions| (**resolutions).clone()) + .unwrap_or_default(); + let mut callable_name_used_signature_keys_for_emit = callable_name_used_signature_keys + .as_ref() + .map(|used_keys| (**used_keys).clone()) + .unwrap_or_default(); + if callable_name_use_facts.generic_trait_used { + callable_name_used_signature_keys_for_emit.extend(callable_name_use_facts.function_arg_signature_keys); + } - // Build unified function registry including imported module functions - let mut unified_registry = ir_program.function_registry.clone(); let mut dependency_ir_programs = Vec::new(); - for (_, dep_ast, _) in &self.dependency_modules { - // For dependencies, use best-effort lowering without type info to - // preserve prior behavior and avoid redundant typechecking. - let mut dep_lowering = AstLowering::new(); + for (dep_name, dep_ast, dep_path_segments) in &self.dependency_modules { + let dep_type_info = { + use crate::frontend::typechecker::TypeChecker; + let mut tc = TypeChecker::new(); + self.configure_typechecker(&mut tc); + match tc.check_with_imports_allow_private(dep_ast, &deps) { + Ok(()) => tc.type_info().clone(), + Err(errs) => return Err(GenerationError::TypeCheck(errs)), + } + }; + let mut dep_lowering = AstLowering::new_with_type_info(dep_type_info); dep_lowering.set_current_source_module_name( - dep_ast - .source_path - .as_deref() - .and_then(crate::frontend::module::logical_module_name_from_source_path), + dep_path_segments + .clone() + .map(|segments| segments.join(".")) + .or_else(|| { + dep_ast + .source_path + .as_deref() + .and_then(crate::frontend::module::logical_module_name_from_source_path) + }), ); + dep_lowering.seed_dependency_trait_decls(&self.dependency_modules); dep_lowering.seed_struct_field_aliases(global_aliases.clone()); let dep_ir = dep_lowering.lower_program(dep_ast)?; - unified_registry.merge(&dep_ir.function_registry); - dependency_ir_programs.push(dep_ir); + let module_path = dep_path_segments + .clone() + .unwrap_or_else(|| vec![(*dep_name).to_string()]); + dependency_ir_programs.push((module_path, dep_ir)); } + let canonical_registry = Self::canonical_registry_for_programs( + dependency_ir_programs + .iter() + .map(|(module_path, dep_ir)| (module_path.as_slice(), dep_ir)), + ); // Emit IR to Rust code let use_emit_service = env::var("INCAN_EMIT_SERVICE").ok().as_deref() == Some("1"); @@ -1229,12 +654,7 @@ impl<'a> IrCodegen<'a> { if self.emit_zen_in_main { inner.set_emit_zen(true); } - inner.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - inner.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - inner.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(inner, &dependency_symbol_metadata); inner.set_needs_serde(self.needs_serde); inner.set_external_rust_functions(self.external_rust_functions.clone()); inner.set_strict_generated_lints(self.strict_generated_lints); @@ -1242,22 +662,22 @@ impl<'a> IrCodegen<'a> { self.apply_ordinal_bridge_config(inner, &ordinal_bridge); inner.set_qualify_union_types_from_crate(qualify_union_types_from_crate); inner.set_generated_union_types(generated_union_types); - for dep_ir in &dependency_ir_programs { + inner.set_canonical_function_registry(canonical_registry.clone()); + inner.set_callable_name_current_module_path(Vec::new()); + inner.set_callable_name_resolutions(callable_name_resolutions_for_emit); + inner.set_callable_name_used_signature_keys(callable_name_used_signature_keys_for_emit); + inner.set_callable_name_local_registry(ir_program.function_registry.clone()); + for (_, dep_ir) in &dependency_ir_programs { inner.seed_dependency_nominal_metadata_from_program(dep_ir); } Ok(svc.emit_program(&ir_program)?) } else { - let mut emitter = IrEmitter::new(&unified_registry); + let mut emitter = IrEmitter::new(&ir_program.function_registry); emitter.set_internal_module_roots(internal_module_roots.clone()); if self.emit_zen_in_main { emitter.set_emit_zen(true); } - emitter.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - emitter.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - emitter.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(&mut emitter, &dependency_symbol_metadata); emitter.set_needs_serde(self.needs_serde); emitter.set_external_rust_functions(self.external_rust_functions.clone()); emitter.set_strict_generated_lints(self.strict_generated_lints); @@ -1265,7 +685,12 @@ impl<'a> IrCodegen<'a> { self.apply_ordinal_bridge_config(&mut emitter, &ordinal_bridge); emitter.set_qualify_union_types_from_crate(qualify_union_types_from_crate); emitter.set_generated_union_types(generated_union_types); - for dep_ir in &dependency_ir_programs { + emitter.set_canonical_function_registry(canonical_registry.clone()); + emitter.set_callable_name_current_module_path(Vec::new()); + emitter.set_callable_name_resolutions(callable_name_resolutions_for_emit); + emitter.set_callable_name_used_signature_keys(callable_name_used_signature_keys_for_emit); + emitter.set_callable_name_local_registry(ir_program.function_registry.clone()); + for (_, dep_ir) in &dependency_ir_programs { emitter.seed_dependency_nominal_metadata_from_program(dep_ir); } Ok(emitter.emit_program(&ir_program)?) @@ -1390,7 +815,7 @@ impl<'a> IrCodegen<'a> { .map(|(name, ast, _)| (*name, *ast)) .collect(); let global_aliases = collect_model_field_aliases(program, &deps); - let dependency_type_metadata = collect_dependency_type_metadata(&self.dependency_modules); + let dependency_symbol_metadata = collect_dependency_symbol_metadata(&self.dependency_modules); let uses_std_ordinal_contract = compilation_imports_std_ordinal_contract(program, &self.dependency_modules); let ordinal_bridge = OrdinalBridgeConfig::for_internal_module(uses_std_ordinal_contract); let dependency_reachable_items = @@ -1443,18 +868,62 @@ impl<'a> IrCodegen<'a> { .collect(); super::trait_bound_inference::propagate_trait_bounds_from_programs(current_ir, &external_programs); } + let all_module_canonical_registry = Self::canonical_registry_for_programs( + lowered_modules + .iter() + .map(|(_, module_path, ir)| (module_path.as_slice(), ir)), + ); let mut shared_union_types = HashMap::new(); for (_, _, ir) in &lowered_modules { shared_union_types.extend(IrEmitter::collect_union_types_from_program(ir)); } // Generate main file after dependency lowering so it can own shared crate-root union wrappers. - let main_code = - self.try_generate_via_ir_with_union_config(program, &internal_roots, shared_union_types, true)?; + let mut callable_name_resolutions = HashMap::new(); + let mut callable_name_used_signature_keys = HashSet::new(); + let mut callable_name_function_arg_signature_keys = HashSet::new(); + let mut generic_callable_name_trait_used = false; + for (_, module_path, ir) in &lowered_modules { + IrEmitter::add_callable_name_resolutions_for_program( + &mut callable_name_resolutions, + module_path.clone(), + ir, + ); + let mut reachable_items = dependency_reachable_items.get(module_path).cloned().unwrap_or_default(); + if let Some(injected_items) = self.externally_reachable_items_by_module.get(module_path) { + reachable_items.extend(injected_items.iter().cloned()); + } + let preserve_public_items = + should_preserve_dependency_public_items(module_path, self.preserve_dependency_public_items); + let callable_name_use_facts = IrEmitter::callable_name_use_facts_for_program( + ir, + &reachable_items, + preserve_public_items, + &dependency_symbol_metadata.error_trait_type_names, + ); + callable_name_used_signature_keys.extend(callable_name_use_facts.signature_keys); + callable_name_function_arg_signature_keys.extend(callable_name_use_facts.function_arg_signature_keys); + generic_callable_name_trait_used |= callable_name_use_facts.generic_trait_used; + } + if generic_callable_name_trait_used { + callable_name_used_signature_keys.extend(callable_name_function_arg_signature_keys); + } + + let main_code = self.try_generate_via_ir_with_union_config( + program, + &internal_roots, + shared_union_types, + true, + Some(&mut callable_name_resolutions), + Some(&mut callable_name_used_signature_keys), + )?; let mut modules = HashMap::new(); for (name, module_path, ir) in &lowered_modules { - let reachable_items = dependency_reachable_items.get(module_path).cloned().unwrap_or_default(); + let mut reachable_items = dependency_reachable_items.get(module_path).cloned().unwrap_or_default(); + if let Some(injected_items) = self.externally_reachable_items_by_module.get(module_path) { + reachable_items.extend(injected_items.iter().cloned()); + } let preserve_public_items = should_preserve_dependency_public_items(module_path, self.preserve_dependency_public_items); let use_emit_service = env::var("INCAN_EMIT_SERVICE").ok().as_deref() == Some("1"); @@ -1464,15 +933,14 @@ impl<'a> IrCodegen<'a> { inner.set_internal_module_roots(internal_roots.clone()); inner.set_preserve_public_items(preserve_public_items); inner.set_externally_reachable_items(reachable_items.clone()); - inner.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - inner.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - inner.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(inner, &dependency_symbol_metadata); inner.set_external_rust_functions(self.external_rust_functions.clone()); inner.set_qualify_union_types_from_crate(true); inner.set_emit_generated_union_definitions(false); + inner.set_canonical_function_registry(all_module_canonical_registry.clone()); + inner.set_callable_name_current_module_path(module_path.clone()); + inner.set_callable_name_resolutions(callable_name_resolutions.clone()); + inner.set_callable_name_used_signature_keys(callable_name_used_signature_keys.clone()); self.apply_ordinal_bridge_config(inner, &ordinal_bridge); for (_, _, dep_ir) in &lowered_modules { inner.seed_dependency_nominal_metadata_from_program(dep_ir); @@ -1483,15 +951,14 @@ impl<'a> IrCodegen<'a> { emitter.set_internal_module_roots(internal_roots.clone()); emitter.set_preserve_public_items(preserve_public_items); emitter.set_externally_reachable_items(reachable_items); - emitter.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - emitter.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - emitter.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(&mut emitter, &dependency_symbol_metadata); emitter.set_external_rust_functions(self.external_rust_functions.clone()); emitter.set_qualify_union_types_from_crate(true); emitter.set_emit_generated_union_definitions(false); + emitter.set_canonical_function_registry(all_module_canonical_registry.clone()); + emitter.set_callable_name_current_module_path(module_path.clone()); + emitter.set_callable_name_resolutions(callable_name_resolutions.clone()); + emitter.set_callable_name_used_signature_keys(callable_name_used_signature_keys.clone()); self.apply_ordinal_bridge_config(&mut emitter, &ordinal_bridge); for (_, _, dep_ir) in &lowered_modules { emitter.seed_dependency_nominal_metadata_from_program(dep_ir); @@ -1578,7 +1045,7 @@ impl<'a> IrCodegen<'a> { .map(|(name, ast, _)| (*name, *ast)) .collect(); let global_aliases = collect_model_field_aliases(program, &deps); - let dependency_type_metadata = collect_dependency_type_metadata(&self.dependency_modules); + let dependency_symbol_metadata = collect_dependency_symbol_metadata(&self.dependency_modules); let uses_std_ordinal_contract = compilation_imports_std_ordinal_contract(program, &self.dependency_modules); let ordinal_bridge = OrdinalBridgeConfig::for_internal_module(uses_std_ordinal_contract); let dependency_reachable_items = @@ -1586,14 +1053,15 @@ impl<'a> IrCodegen<'a> { // Generate module files by path let mut lowered_modules = Vec::new(); - for (name, ast, _) in &self.dependency_modules { - // Find matching path by comparing joined segments with module name - // Module name is path segments joined with "_" (e.g., "db_models") - for path in module_paths { - let path_name = path.join("_"); - if path_name != *name { - continue; - } + for (name, ast, stored_path_segments) in &self.dependency_modules { + let matching_path = if let Some(stored_path_segments) = stored_path_segments { + module_paths.iter().find(|path| *path == stored_path_segments) + } else { + // Legacy callers may still register only a flat module name. Prefer explicit path segments when they + // exist because distinct paths such as `a_b` and `a/b` share the same underscore-joined fallback. + module_paths.iter().find(|path| path.join("_") == *name) + }; + if let Some(path) = matching_path { let module_type_info = { use crate::frontend::typechecker::TypeChecker; let mut tc = TypeChecker::new(); @@ -1613,7 +1081,6 @@ impl<'a> IrCodegen<'a> { // newtypes (e.g., stdlib wrapper types like std.web.request.Query/Path). super::trait_bound_inference::infer_trait_bounds(&mut ir); lowered_modules.push((path.clone(), ir)); - break; } } for idx in 0..lowered_modules.len() { @@ -1631,18 +1098,55 @@ impl<'a> IrCodegen<'a> { .collect(); super::trait_bound_inference::propagate_trait_bounds_from_programs(current_ir, &external_programs); } + let all_module_canonical_registry = + Self::canonical_registry_for_programs(lowered_modules.iter().map(|(path, ir)| (path.as_slice(), ir))); let mut shared_union_types = HashMap::new(); for (_, ir) in &lowered_modules { shared_union_types.extend(IrEmitter::collect_union_types_from_program(ir)); } // Generate main file after dependency lowering so it can own shared crate-root union wrappers. - let main_code = - self.try_generate_via_ir_with_union_config(program, &internal_roots, shared_union_types, true)?; + let mut callable_name_resolutions = HashMap::new(); + let mut callable_name_used_signature_keys = HashSet::new(); + let mut callable_name_function_arg_signature_keys = HashSet::new(); + let mut generic_callable_name_trait_used = false; + for (path, ir) in &lowered_modules { + IrEmitter::add_callable_name_resolutions_for_program(&mut callable_name_resolutions, path.clone(), ir); + let mut reachable_items = dependency_reachable_items.get(path).cloned().unwrap_or_default(); + if let Some(injected_items) = self.externally_reachable_items_by_module.get(path) { + reachable_items.extend(injected_items.iter().cloned()); + } + let preserve_public_items = + should_preserve_dependency_public_items(path, self.preserve_dependency_public_items); + let callable_name_use_facts = IrEmitter::callable_name_use_facts_for_program( + ir, + &reachable_items, + preserve_public_items, + &dependency_symbol_metadata.error_trait_type_names, + ); + callable_name_used_signature_keys.extend(callable_name_use_facts.signature_keys); + callable_name_function_arg_signature_keys.extend(callable_name_use_facts.function_arg_signature_keys); + generic_callable_name_trait_used |= callable_name_use_facts.generic_trait_used; + } + if generic_callable_name_trait_used { + callable_name_used_signature_keys.extend(callable_name_function_arg_signature_keys); + } + + let main_code = self.try_generate_via_ir_with_union_config( + program, + &internal_roots, + shared_union_types, + true, + Some(&mut callable_name_resolutions), + Some(&mut callable_name_used_signature_keys), + )?; let mut modules = HashMap::new(); for (path, ir) in &lowered_modules { - let reachable_items = dependency_reachable_items.get(path).cloned().unwrap_or_default(); + let mut reachable_items = dependency_reachable_items.get(path).cloned().unwrap_or_default(); + if let Some(injected_items) = self.externally_reachable_items_by_module.get(path) { + reachable_items.extend(injected_items.iter().cloned()); + } let preserve_public_items = should_preserve_dependency_public_items(path, self.preserve_dependency_public_items); let use_emit_service = env::var("INCAN_EMIT_SERVICE").ok().as_deref() == Some("1"); @@ -1652,15 +1156,14 @@ impl<'a> IrCodegen<'a> { inner.set_internal_module_roots(internal_roots.clone()); inner.set_preserve_public_items(preserve_public_items); inner.set_externally_reachable_items(reachable_items.clone()); - inner.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - inner.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - inner.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(inner, &dependency_symbol_metadata); inner.set_external_rust_functions(self.external_rust_functions.clone()); inner.set_qualify_union_types_from_crate(true); inner.set_emit_generated_union_definitions(false); + inner.set_canonical_function_registry(all_module_canonical_registry.clone()); + inner.set_callable_name_current_module_path(path.clone()); + inner.set_callable_name_resolutions(callable_name_resolutions.clone()); + inner.set_callable_name_used_signature_keys(callable_name_used_signature_keys.clone()); self.apply_ordinal_bridge_config(inner, &ordinal_bridge); for (_, dep_ir) in &lowered_modules { inner.seed_dependency_nominal_metadata_from_program(dep_ir); @@ -1671,15 +1174,14 @@ impl<'a> IrCodegen<'a> { emitter.set_internal_module_roots(internal_roots.clone()); emitter.set_preserve_public_items(preserve_public_items); emitter.set_externally_reachable_items(reachable_items); - emitter.set_type_module_paths( - dependency_type_metadata.module_paths.clone(), - dependency_type_metadata.ambiguous_type_names.clone(), - ); - emitter.set_dependency_enum_types(dependency_type_metadata.enum_type_names.clone()); - emitter.set_external_error_trait_types(dependency_type_metadata.error_trait_type_names.clone()); + Self::apply_dependency_symbol_metadata(&mut emitter, &dependency_symbol_metadata); emitter.set_external_rust_functions(self.external_rust_functions.clone()); emitter.set_qualify_union_types_from_crate(true); emitter.set_emit_generated_union_definitions(false); + emitter.set_canonical_function_registry(all_module_canonical_registry.clone()); + emitter.set_callable_name_current_module_path(path.clone()); + emitter.set_callable_name_resolutions(callable_name_resolutions.clone()); + emitter.set_callable_name_used_signature_keys(callable_name_used_signature_keys.clone()); self.apply_ordinal_bridge_config(&mut emitter, &ordinal_bridge); for (_, dep_ir) in &lowered_modules { emitter.seed_dependency_nominal_metadata_from_program(dep_ir); @@ -1879,6 +1381,38 @@ def main() -> int: assert!(!code.contains("fn mean"), "{code}"); } + #[test] + fn top_level_keyword_named_callable_alias_uses_raw_identifier_reexport() { + let code = generate( + r#" +pub def modulo_value(value: int) -> int: + return value + +pub mod = alias modulo_value + +def main() -> int: + return mod(10) +"#, + ); + assert!(code.contains("pub fn modulo_value(value: i64) -> i64"), "{code}"); + assert!(code.contains("pub use modulo_value as r#mod;"), "{code}"); + assert!(code.contains("return modulo_value(10);"), "{code}"); + } + + #[test] + fn top_level_alias_to_keyword_named_callable_uses_raw_identifier_target_path() { + let code = generate( + r#" +pub def mod(value: int) -> int: + return value + +pub modulo = alias mod +"#, + ); + assert!(code.contains("pub fn r#mod(value: i64) -> i64"), "{code}"); + assert!(code.contains("pub use r#mod as modulo;"), "{code}"); + } + #[test] fn top_level_qualified_alias_preserves_target_path() { let code = generate( @@ -2119,6 +1653,7 @@ def main() -> None: IrImportItem { name: String::from("Rng"), alias: None, + is_static: false, rust_trait_import: Some(IrRustTraitImport { trait_path: String::from("rand::Rng"), definition_path: None, @@ -2128,6 +1663,7 @@ def main() -> None: IrImportItem { name: String::from("thread_rng"), alias: None, + is_static: false, rust_trait_import: None, }, ], @@ -2232,6 +1768,7 @@ def main() -> None: IrImportItem { name: String::from("AlphaRender"), alias: None, + is_static: false, rust_trait_import: Some(IrRustTraitImport { trait_path: String::from("demo::AlphaRender"), definition_path: None, @@ -2241,6 +1778,7 @@ def main() -> None: IrImportItem { name: String::from("BetaRender"), alias: None, + is_static: false, rust_trait_import: Some(IrRustTraitImport { trait_path: String::from("demo::BetaRender"), definition_path: None, @@ -2337,11 +1875,13 @@ def main() -> None: IrImportItem { name: String::from("Rng"), alias: None, + is_static: false, rust_trait_import: None, }, IrImportItem { name: String::from("thread_rng"), alias: None, + is_static: false, rust_trait_import: None, }, ], @@ -2447,6 +1987,7 @@ def main() -> None: IrImportItem { name: String::from("Digest"), alias: None, + is_static: false, rust_trait_import: Some(IrRustTraitImport { trait_path: String::from("sha2::Digest"), definition_path: Some(String::from("digest::digest::Digest")), @@ -2456,6 +1997,7 @@ def main() -> None: IrImportItem { name: String::from("Sha256"), alias: None, + is_static: false, rust_trait_import: None, }, ], diff --git a/src/backend/ir/codegen/dependency_metadata.rs b/src/backend/ir/codegen/dependency_metadata.rs new file mode 100644 index 000000000..f0519d146 --- /dev/null +++ b/src/backend/ir/codegen/dependency_metadata.rs @@ -0,0 +1,323 @@ +//! Dependency metadata planning for IR code generation. + +use std::collections::{HashMap, HashSet}; + +use crate::frontend::ast::{self, Declaration, Expr, ImportKind, ImportPath, Program}; +use crate::frontend::decorator_resolution; +use crate::frontend::module::canonicalize_source_module_segments; +use crate::frontend::typechecker::stdlib_loader::StdlibAstCache; +use incan_core::lang::stdlib; +use incan_core::lang::traits::{self as core_traits, TraitId}; + +pub(super) fn collect_model_field_aliases( + main: &Program, + deps: &[(&str, &Program)], +) -> HashMap> { + let mut out: HashMap> = HashMap::new(); + + let mut visit = |p: &Program| { + for decl in &p.declarations { + let Declaration::Model(m) = &decl.node else { + continue; + }; + + let mut map: HashMap = HashMap::new(); + for f in &m.fields { + if let Some(alias) = &f.node.metadata.alias { + map.insert(alias.clone(), f.node.name.clone()); + } + } + + if !map.is_empty() { + out.entry(m.name.clone()).or_default().extend(map); + } + } + }; + + visit(main); + for (_, dep) in deps { + visit(dep); + } + + out +} + +/// Resolve a source import path to the generated Rust module path used for dependency emission. +fn generated_module_path_for_source_import(path: &ImportPath, current_module_path: &[String]) -> Option> { + let resolved_segments = if path.parent_levels > 0 { + let keep = current_module_path.len().checked_sub(path.parent_levels)?; + let mut resolved = current_module_path[..keep].to_vec(); + resolved.extend(path.segments.clone()); + resolved + } else { + path.segments.clone() + }; + let mut segments = canonicalize_source_module_segments(&resolved_segments); + + if segments.first().map(String::as_str) == Some(stdlib::STDLIB_ROOT) { + segments[0] = stdlib::INCAN_STD_NAMESPACE.to_string(); + } + + Some(segments) +} + +/// True when a dependency module should keep its public API even if the main module does not import every item. +pub(super) fn should_preserve_dependency_public_items( + module_path: &[String], + preserve_non_stdlib_public_items: bool, +) -> bool { + if matches!( + module_path.first().map(String::as_str), + Some(stdlib::STDLIB_ROOT | stdlib::INCAN_STD_NAMESPACE) + ) { + return true; + } + preserve_non_stdlib_public_items +} + +/// Return whether a function carries the stdlib-backed web route decorator that lowers to a Rust proc-macro attribute. +fn has_web_route_passthrough_decorator( + func: &ast::FunctionDecl, + aliases: &HashMap>, + stdlib_cache: &mut StdlibAstCache, +) -> bool { + func.decorators.iter().any(|decorator| { + let resolved = decorator_resolution::resolve_decorator_path(&decorator.node, aliases); + if resolved.len() < 2 { + return false; + } + let module_segments = &resolved[..resolved.len() - 1]; + let name = &resolved[resolved.len() - 1]; + if name != "route" { + return false; + } + let Some(meta) = stdlib_cache.lookup_function_meta(module_segments, name) else { + return false; + }; + meta.is_rust_extern && meta.rust_module_path.as_deref() == Some("incan_web_macros") + }) +} + +/// Collect dependency-module declarations that must remain reachable from externally visible roots such as imports, +/// ambient logging, and web route registration. +pub(super) fn collect_externally_reachable_items_by_module( + main: &Program, + dependency_modules: &[(&str, &Program, Option>)], +) -> HashMap, HashSet> { + let module_paths: HashSet> = dependency_modules + .iter() + .map(|(name, _, path_segments)| path_segments.clone().unwrap_or_else(|| vec![(*name).to_string()])) + .collect(); + + fn record_imports( + reachable: &mut HashMap, HashSet>, + program: &Program, + current_module_path: &[String], + module_paths: &HashSet>, + ) { + if crate::frontend::surface_semantics::uses_ambient_log_surface(program) { + reachable + .entry(vec!["std".to_string(), "logging".to_string()]) + .or_default() + .insert("get_logger".to_string()); + } + let mut module_import_bindings: HashMap> = HashMap::new(); + for decl in &program.declarations { + let Declaration::Import(import) = &decl.node else { + continue; + }; + match &import.kind { + ImportKind::From { module, items } => { + let Some(module_path) = generated_module_path_for_source_import(module, current_module_path) else { + continue; + }; + let reachable_items = reachable.entry(module_path).or_default(); + for item in items { + reachable_items.insert(item.name.clone()); + } + } + ImportKind::Module(path) => { + let Some(segments) = generated_module_path_for_source_import(path, current_module_path) else { + continue; + }; + if module_paths.contains(&segments) { + if let Some(binding) = import.alias.clone().or_else(|| path.segments.last().cloned()) { + module_import_bindings.insert(binding, segments); + } + continue; + } + let Some(item_name) = segments.last() else { + continue; + }; + for module_path in module_paths { + if segments.len() == module_path.len() + 1 && segments.starts_with(module_path) { + reachable + .entry(module_path.clone()) + .or_default() + .insert(item_name.clone()); + break; + } + } + } + ImportKind::PubLibrary { .. } + | ImportKind::PubFrom { .. } + | ImportKind::RustCrate { .. } + | ImportKind::RustFrom { .. } + | ImportKind::Python(_) => {} + } + } + if !module_import_bindings.is_empty() { + let _ = crate::frontend::ast_walk::any_expr_in_program(program, |expr| { + if let Expr::Field(object, field) = expr + && let Expr::Ident(binding) = &object.node + && let Some(module_path) = module_import_bindings.get(binding) + { + reachable.entry(module_path.clone()).or_default().insert(field.clone()); + } + if let Expr::MethodCall(object, method, _, _) = expr + && let Expr::Ident(binding) = &object.node + && let Some(module_path) = module_import_bindings.get(binding) + { + reachable.entry(module_path.clone()).or_default().insert(method.clone()); + } + false + }); + } + if module_paths.contains(current_module_path) { + let aliases = decorator_resolution::collect_import_aliases(program); + let mut stdlib_cache = StdlibAstCache::new(); + for decl in &program.declarations { + let Declaration::Function(func) = &decl.node else { + continue; + }; + if has_web_route_passthrough_decorator(func, &aliases, &mut stdlib_cache) { + reachable + .entry(current_module_path.to_vec()) + .or_default() + .insert(func.name.clone()); + } + } + } + } + + let mut reachable = HashMap::new(); + record_imports(&mut reachable, main, &[String::from("main")], &module_paths); + for (name, program, path_segments) in dependency_modules { + let module_path = path_segments.clone().unwrap_or_else(|| vec![(*name).to_string()]); + record_imports(&mut reachable, program, &module_path, &module_paths); + } + reachable +} + +/// Dependency symbol facts gathered during codegen setup and reused by module emission. +#[derive(Debug, Clone, Default)] +pub(super) struct DependencySymbolMetadata { + pub(super) module_paths: HashMap>, + pub(super) ambiguous_type_names: HashSet, + pub(super) value_module_paths: HashMap>, + pub(super) ambiguous_value_names: HashSet, + pub(super) enum_type_names: HashSet, + pub(super) error_trait_type_names: HashSet, +} + +/// Collect dependency symbol metadata needed by IR emission for cross-module nominal types and values. +pub(super) fn collect_dependency_symbol_metadata( + deps: &[(&str, &Program, Option>)], +) -> DependencySymbolMetadata { + let mut paths: HashMap> = HashMap::new(); + let mut ambiguous: HashSet = HashSet::new(); + let mut value_paths: HashMap> = HashMap::new(); + let mut ambiguous_values: HashSet = HashSet::new(); + let mut enum_type_names: HashSet = HashSet::new(); + let mut non_enum_type_names: HashSet = HashSet::new(); + let mut error_trait_type_names: HashSet = HashSet::new(); + let error_trait_name = core_traits::as_str(TraitId::Error); + + for (_name, program, path_segments) in deps { + for decl in &program.declarations { + if let Some(segs) = path_segments.as_ref() + && let Some(name) = match &decl.node { + Declaration::Const(c) => Some(&c.name), + Declaration::Static(s) => Some(&s.name), + Declaration::Function(f) => Some(&f.name), + Declaration::Partial(p) => Some(&p.name), + Declaration::Alias(a) => Some(&a.name), + Declaration::Import(_) + | Declaration::Model(_) + | Declaration::Class(_) + | Declaration::Trait(_) + | Declaration::TypeAlias(_) + | Declaration::Newtype(_) + | Declaration::Enum(_) + | Declaration::TestModule(_) + | Declaration::Docstring(_) => None, + } + { + if let Some(existing) = value_paths.get(name) { + if existing != segs { + ambiguous_values.insert(name.clone()); + } + } else { + value_paths.insert(name.clone(), segs.clone()); + } + } + + let type_name = match &decl.node { + Declaration::Model(m) => { + if m.traits.iter().any(|bound| bound.node.name == error_trait_name) { + error_trait_type_names.insert(m.name.clone()); + } + Some((&m.name, false)) + } + Declaration::Class(c) => { + if c.traits.iter().any(|bound| bound.node.name == error_trait_name) { + error_trait_type_names.insert(c.name.clone()); + } + Some((&c.name, false)) + } + Declaration::Enum(e) => Some((&e.name, true)), + Declaration::TypeAlias(a) => Some((&a.name, false)), + Declaration::Newtype(n) => Some((&n.name, false)), + _ => None, + }; + let Some((name, is_enum)) = type_name else { + continue; + }; + + if is_enum { + enum_type_names.insert(name.clone()); + } else { + non_enum_type_names.insert(name.clone()); + } + + let Some(segs) = path_segments.as_ref() else { + continue; + }; + + if let Some(existing) = paths.get(name) { + if existing != segs { + ambiguous.insert(name.clone()); + } + } else { + paths.insert(name.clone(), segs.clone()); + } + } + } + + for name in &ambiguous { + paths.remove(name); + } + for name in &ambiguous_values { + value_paths.remove(name); + } + enum_type_names.retain(|name| !ambiguous.contains(name) && !non_enum_type_names.contains(name)); + + DependencySymbolMetadata { + module_paths: paths, + ambiguous_type_names: ambiguous, + value_module_paths: value_paths, + ambiguous_value_names: ambiguous_values, + enum_type_names, + error_trait_type_names, + } +} diff --git a/src/backend/ir/codegen/ordinal_bridge.rs b/src/backend/ir/codegen/ordinal_bridge.rs new file mode 100644 index 000000000..5512cfff0 --- /dev/null +++ b/src/backend/ir/codegen/ordinal_bridge.rs @@ -0,0 +1,284 @@ +//! OrdinalKey bridge planning for generated IR emission. + +use std::collections::{HashMap, HashSet}; +use std::sync::Arc; + +use crate::frontend::ast::{Declaration, ImportKind, ImportPath, Program}; +use crate::frontend::library_manifest_index::{LibraryManifestIndex, LibraryManifestIndexEntry}; +use crate::library_manifest::{EnumValueExport, EnumValueTypeExport}; +use incan_core::lang::trait_capabilities; + +use crate::backend::ir::decl::{IrEnumValue, IrEnumValueType}; +use crate::backend::ir::emit::{ExternalOrdinalCustomKey, ExternalOrdinalValueEnum}; + +/// Return whether a program imports the stdlib ordinal-map contract. +pub(super) fn imports_std_ordinal_contract(program: &Program) -> bool { + let capability = trait_capabilities::stable_ordinal_key(); + program.declarations.iter().any(|decl| { + let Declaration::Import(import) = &decl.node else { + return false; + }; + match &import.kind { + ImportKind::Module(_) => false, + ImportKind::From { module, items } if import_path_matches_capability(module, capability) => items + .iter() + .any(|item| trait_capabilities::import_triggers_capability(capability, item.name.as_str())), + _ => false, + } + }) +} + +/// Return whether an import path names the module that owns a temporary capability contract. +fn import_path_matches_capability(path: &ImportPath, capability: &trait_capabilities::TraitCapabilityInfo) -> bool { + trait_capabilities::module_path_matches(capability, &path.segments) +} + +/// Return whether any module in the current compilation needs value-enum `OrdinalKey` impls. +pub(super) fn compilation_imports_std_ordinal_contract( + main: &Program, + deps: &[(&str, &Program, Option>)], +) -> bool { + imports_std_ordinal_contract(main) || deps.iter().any(|(_, program, _)| imports_std_ordinal_contract(program)) +} + +/// Collect public scalar value enums from loaded `.incnlib` dependencies. +fn external_ordinal_value_enums(index: Option<&Arc>) -> Vec { + let Some(index) = index else { + return Vec::new(); + }; + let mut out = Vec::new(); + for dependency_key in index.known_libraries() { + let Some(LibraryManifestIndexEntry::Loaded { manifest, metadata }) = index.get(&dependency_key) else { + continue; + }; + for enum_export in &manifest.exports.enums { + let Some(value_type) = enum_export.value_type else { + continue; + }; + let value_type = match value_type { + EnumValueTypeExport::Str => IrEnumValueType::String, + EnumValueTypeExport::Int => IrEnumValueType::Int, + }; + let mut values = Vec::new(); + let mut complete = true; + for variant in &enum_export.variants { + let Some(value) = &variant.value else { + complete = false; + break; + }; + values.push(match value { + EnumValueExport::Str(value) => IrEnumValue::String(value.clone()), + EnumValueExport::Int(value) => IrEnumValue::Int(*value), + }); + } + if !complete { + continue; + } + out.push(ExternalOrdinalValueEnum { + dependency_key: dependency_key.clone(), + name: enum_export.name.clone(), + type_identity: enum_export + .ordinal_type_identity + .clone() + .unwrap_or_else(|| format!("{}.{}", metadata.manifest_name, enum_export.name)), + value_type, + values, + }); + } + } + out +} + +/// Return whether a serialized trait bound names the std `OrdinalKey` capability. +fn type_bound_matches_ordinal_key(bound: &crate::library_manifest::TypeBoundExport) -> bool { + let capability = trait_capabilities::stable_ordinal_key(); + let trait_name = bound + .source_name + .as_deref() + .unwrap_or_else(|| bound.name.rsplit('.').next().unwrap_or(bound.name.as_str())); + if trait_name != capability.trait_name { + return false; + } + let Some(module_path) = &bound.module_path else { + return false; + }; + trait_capabilities::module_path_matches(capability, module_path) +} + +/// Return whether any exported trait adoption satisfies the std `OrdinalKey` contract. +fn export_adopts_ordinal_key( + trait_adoptions: &[crate::library_manifest::TypeBoundExport], + traits: &HashMap, +) -> bool { + trait_adoptions + .iter() + .any(|bound| type_bound_matches_ordinal_key(bound) || trait_bound_extends_ordinal_key(bound, traits)) +} + +/// Return whether a serialized trait bound resolves transitively to std `OrdinalKey`. +fn trait_bound_extends_ordinal_key( + bound: &crate::library_manifest::TypeBoundExport, + traits: &HashMap, +) -> bool { + let mut seen = HashSet::new(); + let mut work = vec![bound.name.as_str()]; + while let Some(name) = work.pop() { + if !seen.insert(name.to_string()) { + continue; + } + let Some(trait_export) = traits.get(name) else { + continue; + }; + for supertrait in &trait_export.supertraits { + if type_bound_matches_ordinal_key(supertrait) { + return true; + } + work.push(supertrait.name.as_str()); + } + } + false +} + +/// Return lookup keys for a manifest trait export, including its original source name when reexported under an alias. +fn trait_export_lookup_keys(trait_export: &crate::library_manifest::TraitExport) -> Vec { + let mut keys = vec![trait_export.name.clone()]; + if let Some(source_name) = &trait_export.source_name + && source_name != &trait_export.name + { + keys.push(source_name.clone()); + } + keys +} + +/// Return whether a manifest method set exposes a source method or its generated alias. +fn export_methods_include(methods: &[crate::library_manifest::MethodExport], name: &str) -> bool { + methods + .iter() + .any(|method| method.name == name || method.alias_of.as_deref() == Some(name)) +} + +/// Build custom-key bridge metadata for one exported concrete type when it adopts `OrdinalKey`. +fn external_ordinal_custom_key( + dependency_key: &str, + name: &str, + type_params: &[crate::library_manifest::TypeParamExport], + trait_adoptions: &[crate::library_manifest::TypeBoundExport], + methods: &[crate::library_manifest::MethodExport], + traits: &HashMap, +) -> Option { + if !type_params.is_empty() || !export_adopts_ordinal_key(trait_adoptions, traits) { + return None; + } + let hooks = trait_capabilities::stable_ordinal_key().bridge_hooks?; + Some(ExternalOrdinalCustomKey { + dependency_key: dependency_key.to_string(), + name: name.to_string(), + has_ordinal_hash: export_methods_include(methods, hooks.hash_method), + has_ordinal_bytes_equal: export_methods_include(methods, hooks.bytes_equal_method), + }) +} + +/// Collect public user-authored `OrdinalKey` adopters from loaded `.incnlib` dependencies. +fn external_ordinal_custom_keys(index: Option<&Arc>) -> Vec { + let Some(index) = index else { + return Vec::new(); + }; + let mut out = Vec::new(); + for dependency_key in index.known_libraries() { + let Some(LibraryManifestIndexEntry::Loaded { manifest, .. }) = index.get(&dependency_key) else { + continue; + }; + let traits = manifest + .exports + .traits + .iter() + .flat_map(|trait_export| { + trait_export_lookup_keys(trait_export) + .into_iter() + .map(move |key| (key, trait_export)) + }) + .collect::>(); + for model in &manifest.exports.models { + if let Some(key) = external_ordinal_custom_key( + &dependency_key, + &model.name, + &model.type_params, + &model.trait_adoptions, + &model.methods, + &traits, + ) { + out.push(key); + } + } + for class in &manifest.exports.classes { + if let Some(key) = external_ordinal_custom_key( + &dependency_key, + &class.name, + &class.type_params, + &class.trait_adoptions, + &class.methods, + &traits, + ) { + out.push(key); + } + } + for newtype in &manifest.exports.newtypes { + if let Some(key) = external_ordinal_custom_key( + &dependency_key, + &newtype.name, + &newtype.type_params, + &newtype.trait_adoptions, + &newtype.methods, + &traits, + ) { + out.push(key); + } + } + for enum_export in &manifest.exports.enums { + if enum_export.value_type.is_some() { + continue; + } + if let Some(key) = external_ordinal_custom_key( + &dependency_key, + &enum_export.name, + &enum_export.type_params, + &enum_export.trait_adoptions, + &enum_export.methods, + &traits, + ) { + out.push(key); + } + } + } + out +} + +#[derive(Debug, Clone)] +pub(super) struct OrdinalBridgeConfig { + pub(super) emit_std_ordinal_value_enum_impls: bool, + pub(super) external_value_enums: Vec, + pub(super) external_custom_keys: Vec, +} + +impl OrdinalBridgeConfig { + /// Build a bridge configuration for generated internal modules. + pub(super) fn for_internal_module(uses_std_ordinal_contract: bool) -> Self { + Self { + emit_std_ordinal_value_enum_impls: uses_std_ordinal_contract, + external_value_enums: Vec::new(), + external_custom_keys: Vec::new(), + } + } + + /// Build a bridge configuration for crate-root emission where dependency adapters live. + pub(super) fn for_crate_root(uses_std_ordinal_contract: bool, index: Option<&Arc>) -> Self { + if !uses_std_ordinal_contract { + return Self::for_internal_module(false); + } + Self { + emit_std_ordinal_value_enum_impls: true, + external_value_enums: external_ordinal_value_enums(index), + external_custom_keys: external_ordinal_custom_keys(index), + } + } +} diff --git a/src/backend/ir/codegen/serde_activation.rs b/src/backend/ir/codegen/serde_activation.rs new file mode 100644 index 000000000..04ae5c47f --- /dev/null +++ b/src/backend/ir/codegen/serde_activation.rs @@ -0,0 +1,139 @@ +//! Serde derive and JSON activation planning for IR code generation. + +use crate::frontend::ast::{Declaration, Program}; +use crate::frontend::decorator_resolution; +use incan_core::lang::decorators::{self, DecoratorId}; +use incan_core::lang::stdlib; + +const SERDE_SERIALIZE_DERIVE: &str = "serde::Serialize"; +const SERDE_DESERIALIZE_DERIVE: &str = "serde::Deserialize"; + +/// Return whether any loaded module derives serde serialize or deserialize through resolved JSON derive imports. +pub(super) fn collect_serde_derives(main: &Program, deps: &[(&str, &Program)]) -> (bool, bool) { + let mut has_serialize = false; + let mut has_deserialize = false; + + let mut visit = |program: &Program| { + let import_aliases = decorator_resolution::collect_import_aliases(program); + for decl in &program.declarations { + let decorators = match &decl.node { + Declaration::Model(m) => Some(&m.decorators), + Declaration::Class(c) => Some(&c.decorators), + Declaration::Enum(e) => Some(&e.decorators), + _ => None, + }; + let Some(decorators) = decorators else { + continue; + }; + for dec in decorators { + if decorators::from_str(dec.node.name.as_str()) != Some(DecoratorId::Derive) { + continue; + } + for arg in &dec.node.args { + let crate::frontend::ast::DecoratorArg::Positional(expr) = arg else { + continue; + }; + let crate::frontend::ast::Expr::Ident(name) = &expr.node else { + continue; + }; + let resolved = import_aliases + .get(name) + .cloned() + .unwrap_or_else(|| vec![name.to_string()]); + match stdlib::stdlib_json_trait_id_from_path(&resolved) { + Some(stdlib::StdlibJsonTraitId::Serialize) => { + has_serialize = true; + } + Some(stdlib::StdlibJsonTraitId::Deserialize) => { + has_deserialize = true; + } + None if stdlib::is_stdlib_json_trait_module_path(&resolved) => { + has_serialize = true; + has_deserialize = true; + } + None => match resolved.as_slice() { + [serde, trait_name] if serde == "serde" && trait_name == "Serialize" => { + has_serialize = true; + } + [serde, trait_name] if serde == "serde" && trait_name == "Deserialize" => { + has_deserialize = true; + } + _ => {} + }, + } + } + } + } + }; + + visit(main); + for (_, dep) in deps { + visit(dep); + } + + if !has_serialize && !has_deserialize { + let serde_used = crate::backend::ir::scanners::detect_serde_usage(main) + || deps + .iter() + .any(|(_, program)| crate::backend::ir::scanners::detect_serde_usage(program)); + if serde_used { + has_serialize = true; + } + } + + (has_serialize, has_deserialize) +} + +/// Add serde derives to generated newtypes when the current program needs serde support. +pub(super) fn add_serde_to_newtypes( + ir_program: &mut crate::backend::ir::IrProgram, + add_serialize: bool, + add_deserialize: bool, +) { + use crate::backend::ir::decl::IrDeclKind; + use crate::backend::ir::types::IrType; + + fn is_conservative_serde_safe_newtype_inner(ty: &IrType) -> bool { + match ty { + IrType::Unit + | IrType::Bool + | IrType::Int + | IrType::Float + | IrType::String + | IrType::Bytes + | IrType::StaticStr + | IrType::StaticBytes + | IrType::FrozenStr + | IrType::FrozenBytes + | IrType::StrRef => true, + IrType::List(inner) | IrType::Set(inner) | IrType::Option(inner) => { + is_conservative_serde_safe_newtype_inner(inner) + } + IrType::Dict(key, value) | IrType::Result(key, value) => { + is_conservative_serde_safe_newtype_inner(key) && is_conservative_serde_safe_newtype_inner(value) + } + IrType::Tuple(items) => items.iter().all(is_conservative_serde_safe_newtype_inner), + _ => false, + } + } + + for decl in &mut ir_program.declarations { + if let IrDeclKind::Struct(s) = &mut decl.kind + && s.fields.len() == 1 + && s.fields[0].name == "0" + { + if !s.type_params.is_empty() { + continue; + } + if !is_conservative_serde_safe_newtype_inner(&s.fields[0].ty) { + continue; + } + if add_serialize && !s.derives.iter().any(|d| d == SERDE_SERIALIZE_DERIVE) { + s.derives.push(SERDE_SERIALIZE_DERIVE.to_string()); + } + if add_deserialize && !s.derives.iter().any(|d| d == SERDE_DESERIALIZE_DERIVE) { + s.derives.push(SERDE_DESERIALIZE_DERIVE.to_string()); + } + } + } +} diff --git a/src/backend/ir/conversions.rs b/src/backend/ir/conversions.rs index 368a092c1..25e2abfd3 100644 --- a/src/backend/ir/conversions.rs +++ b/src/backend/ir/conversions.rs @@ -171,6 +171,7 @@ use super::decl::FunctionParam; use super::expr::{BinOp, VarAccess}; +use super::reference_shape::expr_has_rust_reference_shape; use super::types::Mutability; use super::{IrExpr, IrExprKind, IrType, TypedExpr}; use crate::numeric_adapters::{ir_type_to_numeric_ty, numeric_op_from_ir, pow_exponent_kind_from_ir}; @@ -518,16 +519,16 @@ fn determine_owned_storage_conversion(expr: &IrExpr, target_ty: Option<&IrType>) match (&expr.kind, target_ty) { (IrExprKind::String(_), Some(IrType::String)) => Conversion::ToString, (IrExprKind::StaticRead { .. }, Some(IrType::String | IrType::Generic(_))) - if matches!(expr.ty, IrType::StaticStr) => + if is_borrowed_string_like_type(&expr.ty) => { Conversion::ToString } - (IrExprKind::StaticRead { .. }, None) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, - (_, Some(IrType::String)) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (IrExprKind::StaticRead { .. }, None) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, + (_, Some(IrType::String)) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, (IrExprKind::String(_), Some(IrType::Generic(_))) => Conversion::ToString, - (_, Some(IrType::Generic(_))) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, Some(IrType::Generic(_))) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, (IrExprKind::String(_), None) => Conversion::ToString, - (_, None) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, None) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, _ if borrowed_expr_needs_owned_materialization(expr, target_ty) => Conversion::Clone, (IrExprKind::Var { access, .. }, Some(IrType::String)) if matches!(expr.ty, IrType::String) => match access { VarAccess::Move => Conversion::None, @@ -594,6 +595,16 @@ fn is_result_like_type(ty: &IrType) -> bool { } } +/// Return whether a source value has Rust borrowed/static string shape while representing Incan `str`. +fn is_borrowed_string_like_type(ty: &IrType) -> bool { + matches!(ty, IrType::StaticStr | IrType::StrRef | IrType::FrozenStr) +} + +/// Return whether an owned Incan sink needs borrowed/static string materialization. +fn borrowed_string_like_needs_owned_string(source_ty: &IrType, target_ty: Option<&IrType>) -> bool { + is_borrowed_string_like_type(source_ty) && matches!(target_ty, None | Some(IrType::String | IrType::Generic(_))) +} + /// Whether a value type came from Rust interop and can reasonably cross an Incan `str` boundary via `ToString`. /// /// Lowering maps `ResolvedType::RustPath` to `IrType::Struct(path)`, so the stable signal left in IR is a Rust-style @@ -684,23 +695,23 @@ pub fn determine_conversion(expr: &IrExpr, target_ty: Option<&IrType>, context: (IrExprKind::String(_), Some(IrType::String)) => Conversion::ToString, // Static const reads still represent Incan `str` at ordinary call sites. (IrExprKind::StaticRead { .. }, Some(IrType::String | IrType::Generic(_))) - if matches!(expr.ty, IrType::StaticStr) => + if is_borrowed_string_like_type(&expr.ty) => { Conversion::ToString } - (IrExprKind::StaticRead { .. }, None) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, - // Const `str` values lower as `&'static str` but still follow Incan owned-string semantics at call - // sites. - (_, Some(IrType::String)) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (IrExprKind::StaticRead { .. }, None) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, + // Const/imported `str` values can lower as borrowed/static Rust string shapes but still follow Incan + // owned-string semantics at call sites. + (_, Some(IrType::String)) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, // String literal to generic type param (e.g. assert_eq[T]) → owned String. // Typechecker constrains `T`; this keeps Incan `str` semantics in generic calls. (IrExprKind::String(_), Some(IrType::Generic(_))) => Conversion::ToString, // Generic `T` instantiated with Incan `str` must still materialize to owned `String`. - (_, Some(IrType::Generic(_))) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, Some(IrType::Generic(_))) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, // String literal with unknown target (enum variants, etc.) → .to_string() (IrExprKind::String(_), None) => Conversion::ToString, // Const `str` values need the same owned-string materialization when the target is inferred. - (_, None) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, None) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, // Borrowed method-chain results such as `box.as_ref()` must materialize owned values at Incan call // boundaries. _ if borrowed_expr_needs_owned_materialization(expr, target_ty) => Conversion::Clone, @@ -739,9 +750,12 @@ pub fn determine_conversion(expr: &IrExpr, target_ty: Option<&IrType>, context: match (&expr.kind, target_ty) { // String literal → .to_string() (IrExprKind::String(_), _) => Conversion::ToString, - (IrExprKind::StaticRead { .. }, _) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, - // Const `str` values remain owned `str` at the Incan surface even inside return-context calls. - (_, _) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (IrExprKind::StaticRead { .. }, _) if borrowed_string_like_needs_owned_string(&expr.ty, target_ty) => { + Conversion::ToString + } + // Const/imported `str` values remain owned `str` at the Incan surface even inside return-context + // calls. + (_, _) if borrowed_string_like_needs_owned_string(&expr.ty, target_ty) => Conversion::ToString, _ if borrowed_expr_needs_owned_materialization(expr, target_ty) => Conversion::Clone, _ if rust_value_needs_stringification(expr, target_ty) => Conversion::ToString, @@ -793,14 +807,22 @@ pub fn determine_conversion(expr: &IrExpr, target_ty: Option<&IrType>, context: (IrExprKind::Field { .. }, Some(IrType::String)) if matches!(expr.ty, IrType::String) => { Conversion::Clone } - (IrExprKind::Var { .. }, _) if matches!(expr.ty, IrType::String) => Conversion::Borrow, - (IrExprKind::Field { .. }, None) if matches!(expr.ty, IrType::String) => Conversion::Borrow, - (_, Some(IrType::Ref(_))) if !matches!(expr.ty, IrType::Ref(_) | IrType::RefMut(_)) => { - Conversion::Borrow + (IrExprKind::Var { .. }, _) if matches!(expr.ty, IrType::String) => { + if expr_has_rust_reference_shape(expr) { + Conversion::None + } else { + Conversion::Borrow + } } - (_, Some(IrType::RefMut(_))) if !matches!(expr.ty, IrType::Ref(_) | IrType::RefMut(_)) => { - Conversion::MutBorrow + (IrExprKind::Field { .. }, None) if matches!(expr.ty, IrType::String) => { + if expr_has_rust_reference_shape(expr) { + Conversion::None + } else { + Conversion::Borrow + } } + (_, Some(IrType::Ref(_))) if !expr_has_rust_reference_shape(expr) => Conversion::Borrow, + (_, Some(IrType::RefMut(_))) if !expr_has_rust_reference_shape(expr) => Conversion::MutBorrow, // Rust adapter leaves commonly accept borrowed handles (`&Sender`, `&Mutex`, ...). // When metadata is unavailable, do not move non-Copy wrapper fields out of `&self`. (IrExprKind::Field { .. }, None) @@ -828,11 +850,11 @@ pub fn determine_conversion(expr: &IrExpr, target_ty: Option<&IrType>, context: // String literal assigned to String variable → .to_string() (IrExprKind::String(_), Some(IrType::String)) => Conversion::ToString, (IrExprKind::StaticRead { .. }, Some(IrType::String | IrType::Generic(_))) - if matches!(expr.ty, IrType::StaticStr) => + if is_borrowed_string_like_type(&expr.ty) => { Conversion::ToString } - (_, Some(IrType::String)) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, Some(IrType::String)) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, _ if borrowed_expr_needs_owned_materialization(expr, target_ty) => Conversion::Clone, (IrExprKind::Field { .. }, _) if matches!(expr.ty, IrType::String) && field_read_needs_owned_materialization(expr) => @@ -851,10 +873,10 @@ pub fn determine_conversion(expr: &IrExpr, target_ty: Option<&IrType>, context: match (&expr.kind, target_ty) { // String literal returned when function returns String → .to_string() (IrExprKind::String(_), Some(IrType::String)) => Conversion::ToString, - (IrExprKind::StaticRead { .. }, Some(IrType::String)) if matches!(expr.ty, IrType::StaticStr) => { + (IrExprKind::StaticRead { .. }, Some(IrType::String)) if is_borrowed_string_like_type(&expr.ty) => { Conversion::ToString } - (_, Some(IrType::String)) if matches!(expr.ty, IrType::StaticStr) => Conversion::ToString, + (_, Some(IrType::String)) if is_borrowed_string_like_type(&expr.ty) => Conversion::ToString, _ if borrowed_expr_needs_owned_materialization(expr, target_ty) => Conversion::Clone, // Non-Copy vars can move on last use; otherwise materialize an owned return value. (IrExprKind::Var { access, .. }, _) if !expr.ty.is_copy() => match access { @@ -925,11 +947,11 @@ pub(crate) fn determine_conversion_for_incan_call( ) { match target_ty { Some(IrType::Ref(_)) => match &expr.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Conversion::None, + _ if expr_has_rust_reference_shape(expr) => return Conversion::None, _ => return Conversion::Borrow, }, Some(IrType::RefMut(_)) => match &expr.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Conversion::None, + _ if expr_has_rust_reference_shape(expr) => return Conversion::None, _ => return Conversion::MutBorrow, }, _ => {} @@ -949,7 +971,7 @@ pub(crate) fn determine_conversion_for_incan_call( mod tests { use super::*; use crate::backend::ir::decl::FunctionParam; - use crate::backend::ir::expr::{VarAccess, VarRefKind}; + use crate::backend::ir::expr::{MethodCallArgPolicy, VarAccess, VarRefKind}; use crate::backend::ir::types::Mutability; // === IncanFunctionArg Tests === @@ -1066,6 +1088,22 @@ mod tests { assert_eq!(conv, Conversion::ToString); } + #[test] + fn test_incan_function_frozen_str_var_to_string() { + let expr = IrExpr::new( + IrExprKind::Var { + name: "s".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::FrozenStr, + ); + let target = IrType::String; + + let conv = determine_conversion(&expr, Some(&target), ConversionContext::IncanFunctionArg); + assert_eq!(conv, Conversion::ToString); + } + #[test] fn test_incan_function_static_str_var_to_generic() { let expr = IrExpr::new( @@ -1082,6 +1120,22 @@ mod tests { assert_eq!(conv, Conversion::ToString); } + #[test] + fn test_assignment_frozen_str_var_to_string() { + let expr = IrExpr::new( + IrExprKind::Var { + name: "s".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::FrozenStr, + ); + let target = IrType::String; + + let conv = determine_conversion(&expr, Some(&target), ConversionContext::Assignment); + assert_eq!(conv, Conversion::ToString); + } + #[test] fn test_incan_function_rust_path_value_to_string_param() { let expr = IrExpr::new( @@ -1345,6 +1399,44 @@ mod tests { assert_eq!(conv, Conversion::Borrow); } + #[test] + fn test_external_function_as_slice_arg_does_not_double_borrow() { + let expr = IrExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(IrExpr::new( + IrExprKind::Var { + name: "data".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Bytes, + )), + method: "as_slice".to_string(), + dispatch: None, + type_args: Vec::new(), + args: Vec::new(), + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Bytes, + ); + + let conv = determine_conversion(&expr, None, ConversionContext::ExternalFunctionArg); + assert_eq!( + conv, + Conversion::None, + "an explicit as_slice() argument is already a Rust borrow boundary" + ); + + let target = IrType::Ref(Box::new(IrType::Bytes)); + let conv = determine_conversion(&expr, Some(&target), ConversionContext::ExternalFunctionArg); + assert_eq!( + conv, + Conversion::None, + "an explicit as_slice() argument must not become &&[u8] for ref targets" + ); + } + #[test] fn test_external_function_string_var_with_by_value_target_does_not_borrow() { let expr = IrExpr::new( diff --git a/src/backend/ir/decl.rs b/src/backend/ir/decl.rs index 33342ef48..be9f80276 100644 --- a/src/backend/ir/decl.rs +++ b/src/backend/ir/decl.rs @@ -57,6 +57,8 @@ pub enum IrDeclKind { visibility: Visibility, name: String, target_path: Vec, + target_origin: Option, + target_qualifier: Option, }, /// Constant @@ -171,6 +173,12 @@ pub struct IrRustTraitImport { pub struct IrImportItem { pub name: String, pub alias: Option, + /// Whether this import item binds an Incan `static` storage cell. + /// + /// Static declarations use Rust global naming in generated code, so imported static items must emit the provider's + /// static identifier and, when aliased, the local static identifier instead of treating the source spelling as an + /// ordinary Rust value binding. + pub is_static: bool, /// Metadata provided when this item is a Rust trait import. /// /// Extension-trait imports can be used by Rust method lookup without appearing as identifiers in emitted tokens. diff --git a/src/backend/ir/emit/consts.rs b/src/backend/ir/emit/consts.rs index aa356e0cd..5710b7fb3 100644 --- a/src/backend/ir/emit/consts.rs +++ b/src/backend/ir/emit/consts.rs @@ -37,37 +37,11 @@ impl<'a> IrEmitter<'a> { /// /// Everything else is rejected with an actionable error. pub(super) fn validate_const_emittable(&self, name: &str, ty: &IrType, value: &TypedExpr) -> Result<(), EmitError> { - /// Return whether an IR type can appear in a Rust `const` initializer emitted by RFC 008. - fn ok_ty(ty: &IrType) -> bool { - match ty { - IrType::Int - | IrType::Numeric(_) - | IrType::Float - | IrType::Bool - | IrType::StaticStr - | IrType::StaticBytes - | IrType::FrozenStr - | IrType::FrozenBytes => true, - IrType::Struct(_) => true, - IrType::Tuple(items) => items.iter().all(ok_ty), - IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenList) => { - args.first().map(ok_ty).unwrap_or(false) - } - IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenSet) => { - args.first().map(ok_ty).unwrap_or(false) - } - IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenDict) => { - args.first().map(ok_ty).unwrap_or(false) && args.get(1).map(ok_ty).unwrap_or(false) - } - _ => false, - } - } - - if !ok_ty(ty) { + if !self.const_type_emittable(ty) { let ty_name = ty.rust_name(); return Err(EmitError::Unsupported(format!( "const '{}' of type '{}' is not representable as a Rust const.\n\ - Allowed: int/exact-width numeric/float/bool/&'static str/&'static [u8]/tuples, FrozenList/Set/Dict with allowed element types.\n\ + Allowed: int/exact-width numeric/float/bool/&'static str/&'static [u8]/tuples, Option, const-safe models, FrozenList/Set/Dict with allowed element types.\n\ Consider computing at runtime or simplifying the const.", name, ty_name ))); @@ -76,6 +50,64 @@ impl<'a> IrEmitter<'a> { Self::validate_const_expr_kind(&value.kind) } + /// Return whether an IR type can appear in a Rust `const` initializer emitted by RFC 008. + fn const_type_emittable(&self, ty: &IrType) -> bool { + let mut seen_structs = std::collections::HashSet::new(); + self.const_type_emittable_inner(ty, &mut seen_structs) + } + + fn const_type_emittable_inner(&self, ty: &IrType, seen_structs: &mut std::collections::HashSet) -> bool { + match ty { + IrType::Int + | IrType::Numeric(_) + | IrType::Float + | IrType::Bool + | IrType::StaticStr + | IrType::StaticBytes + | IrType::FrozenStr + | IrType::FrozenBytes => true, + IrType::Option(inner) => self.const_type_emittable_inner(inner, seen_structs), + IrType::Struct(name) => self.const_struct_type_emittable(name, seen_structs), + IrType::Tuple(items) => items + .iter() + .all(|item| self.const_type_emittable_inner(item, seen_structs)), + IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenList) => args + .first() + .map(|arg| self.const_type_emittable_inner(arg, seen_structs)) + .unwrap_or(false), + IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenSet) => args + .first() + .map(|arg| self.const_type_emittable_inner(arg, seen_structs)) + .unwrap_or(false), + IrType::NamedGeneric(name, args) if name == collections::as_str(CollectionTypeId::FrozenDict) => { + args.first() + .map(|arg| self.const_type_emittable_inner(arg, seen_structs)) + .unwrap_or(false) + && args + .get(1) + .map(|arg| self.const_type_emittable_inner(arg, seen_structs)) + .unwrap_or(false) + } + _ => false, + } + } + + fn const_struct_type_emittable(&self, name: &str, seen_structs: &mut std::collections::HashSet) -> bool { + if !seen_structs.insert(name.to_string()) { + return false; + } + let emittable = self.struct_constructor_metadata.get(name).is_some_and(|variants| { + variants.iter().any(|metadata| { + metadata + .field_types + .values() + .all(|field_ty| self.const_type_emittable_inner(field_ty, seen_structs)) + }) + }); + seen_structs.remove(name); + emittable + } + /// RFC 008 const expression shape check (defensive backend guard). /// /// Frontend const-eval should already reject non-const expressions, but this @@ -142,8 +174,11 @@ impl<'a> IrEmitter<'a> { } Ok(()) } - K::Struct { fields, .. } if fields.len() == 1 && fields[0].0.is_empty() => { - Self::validate_const_expr_kind(&fields[0].1.kind) + K::Struct { fields, .. } => { + for (_, field_value) in fields { + Self::validate_const_expr_kind(&field_value.kind)?; + } + Ok(()) } K::Call { diff --git a/src/backend/ir/emit/decls/functions.rs b/src/backend/ir/emit/decls/functions.rs index 0ae0beeaa..2b0665b02 100644 --- a/src/backend/ir/emit/decls/functions.rs +++ b/src/backend/ir/emit/decls/functions.rs @@ -274,7 +274,7 @@ impl<'a> IrEmitter<'a> { } IrExprKind::Format { parts } => { for part in parts { - if let super::super::super::expr::FormatPart::Expr(expr) = part { + if let super::super::super::expr::FormatPart::Expr { expr, .. } = part { Self::rewrite_borrowed_param_types_in_expr(expr, borrowed); } } @@ -1483,7 +1483,7 @@ impl<'a> IrEmitter<'a> { } IrExprKind::Format { parts } => { for part in parts { - if let super::super::super::expr::FormatPart::Expr(expr) = part { + if let super::super::super::expr::FormatPart::Expr { expr, .. } = part { Self::collect_expr_used_names(expr, param_names, shadowed_names, used_names); } } diff --git a/src/backend/ir/emit/decls/impls.rs b/src/backend/ir/emit/decls/impls.rs index 63c34f745..08a262abe 100644 --- a/src/backend/ir/emit/decls/impls.rs +++ b/src/backend/ir/emit/decls/impls.rs @@ -176,7 +176,7 @@ impl<'a> IrEmitter<'a> { }) .map(|m| self.emit_trait_method(m)) .collect::>()?; - if Self::is_serde_serialize_trait_name(trait_name) + if incan_core::lang::stdlib::is_stdlib_json_serialize_trait_name(trait_name) && !impl_block.methods.iter().any(|method| method.name == "to_json") { trait_methods.push(quote! { @@ -185,7 +185,7 @@ impl<'a> IrEmitter<'a> { } }); } - if Self::is_serde_deserialize_trait_name(trait_name) + if incan_core::lang::stdlib::is_stdlib_json_deserialize_trait_name(trait_name) && !impl_block.methods.iter().any(|method| method.name == "from_json") { trait_methods.push(quote! { @@ -250,22 +250,6 @@ impl<'a> IrEmitter<'a> { }) } - /// Return whether a trait impl target names the stdlib JSON serialization trait or an imported alias of it. - fn is_serde_serialize_trait_name(trait_name: &str) -> bool { - matches!( - trait_name, - "Serialize" | "JsonSerialize" | "json.Serialize" | "std.serde.json.Serialize" - ) - } - - /// Return whether a trait impl target names the stdlib JSON deserialization trait or an imported alias of it. - fn is_serde_deserialize_trait_name(trait_name: &str) -> bool { - matches!( - trait_name, - "Deserialize" | "JsonDeserialize" | "json.Deserialize" | "std.serde.json.Deserialize" - ) - } - /// Return the final path segment of a trait name. fn trait_short_name(trait_name: &str) -> &str { trait_name diff --git a/src/backend/ir/emit/decls/mod.rs b/src/backend/ir/emit/decls/mod.rs index bfbf5d2cc..5d7b73071 100644 --- a/src/backend/ir/emit/decls/mod.rs +++ b/src/backend/ir/emit/decls/mod.rs @@ -21,7 +21,7 @@ mod mutation_scan; mod structures; use proc_macro2::{Literal, TokenStream}; -use quote::{format_ident, quote}; +use quote::quote; use incan_core::lang::stdlib; @@ -61,7 +61,7 @@ impl<'a> IrEmitter<'a> { interop_edges: _, } => { let vis = self.emit_visibility(visibility); - let name_ident = format_ident!("{}", name); + let name_ident = Self::rust_ident(name); let ty_tokens = self.emit_type(ty); let generics = self.emit_type_params(type_params); Ok(quote! { @@ -72,17 +72,13 @@ impl<'a> IrEmitter<'a> { visibility, name, target_path, + target_origin, + target_qualifier, } => { let vis = self.emit_visibility(visibility); - let name_ident = format_ident!("{}", name); - let target_segments = target_path - .iter() - .map(|segment| { - let ident = format_ident!("{}", segment); - quote! { #ident } - }) - .collect::>(); - let target = join_path_tokens(&target_segments); + let name_ident = Self::rust_ident(name); + let target = + self.emit_symbol_alias_target_path(target_origin.as_ref(), target_qualifier.as_ref(), target_path); Ok(quote! { #vis use #target as #name_ident; }) @@ -149,7 +145,7 @@ impl<'a> IrEmitter<'a> { self.validate_const_emittable(name, ty, value)?; let vis = self.emit_visibility(visibility); - let name_ident = format_ident!("{}", name); + let name_ident = Self::rust_ident(name); let ty_tokens = self.emit_type(ty); let value_tokens = self.emit_const_value_for_type(ty, value)?; @@ -221,6 +217,7 @@ impl<'a> IrEmitter<'a> { let elems = elems?; Ok(quote! { (#(#elems),*) }) } + (T::Struct(_), IrExprKind::Struct { name, fields }) => self.emit_const_struct_value(name, fields), (T::FrozenStr, IrExprKind::String(s)) => Ok(quote! { incan_stdlib::frozen::FrozenStr::new(#s) }), (T::FrozenBytes, IrExprKind::Bytes(bytes)) => { let lit = Literal::byte_string(bytes); @@ -230,8 +227,161 @@ impl<'a> IrEmitter<'a> { } } + /// Emit a struct/model literal in a Rust const initializer without applying runtime ownership conversions. + fn emit_const_struct_value( + &self, + name: &str, + fields: &[(String, super::super::TypedExpr)], + ) -> Result { + let n = Self::rust_ident(name); + let Some(metadata) = self.struct_constructor_metadata_for_fields(name, fields) else { + let field_tokens: Result, EmitError> = fields + .iter() + .map(|(field_name, field_value)| { + let field_ident = Self::rust_ident(field_name); + let value = self.emit_const_value_for_type(&field_value.ty, field_value)?; + Ok(quote! { #field_ident: #value }) + }) + .collect(); + let field_tokens = field_tokens?; + return Ok(quote! { #n { #(#field_tokens),* } }); + }; + + let mut provided: std::collections::HashMap<&str, &super::super::TypedExpr> = std::collections::HashMap::new(); + for (field_name, field_value) in fields { + if let Some(canonical) = metadata.canonical_field_name(field_name) { + provided.insert(canonical, field_value); + } + } + + let mut out_fields = Vec::new(); + for field_name in &metadata.fields { + let field_ident = Self::rust_ident(field_name); + let Some(target_ty) = metadata.field_types.get(field_name) else { + return Err(EmitError::Unsupported(format!( + "missing field type metadata for const field '{}.{}'", + name, field_name + ))); + }; + let Some(field_value) = provided.get(field_name.as_str()) else { + return Err(EmitError::Unsupported(format!( + "const model constructor '{}' must provide field '{}' explicitly", + name, field_name + ))); + }; + let value = self.emit_const_value_for_type(target_ty, field_value)?; + out_fields.push(quote! { #field_ident: #value }); + } + + Ok(quote! { #n { #(#out_fields),* } }) + } + // ---- Import emission ---- + /// Return whether an import path refers to the source-authored Incan stdlib namespace. + pub(super) fn is_incan_source_stdlib_import( + origin: &IrImportOrigin, + qualifier: &IrImportQualifier, + path: &[String], + ) -> bool { + !matches!(origin, IrImportOrigin::PubLibrary { .. }) + && !matches!(qualifier, IrImportQualifier::None) + && stdlib::is_any_stdlib_path(path) + } + + /// Convert an IR import path into Rust path segments using the same qualification rules for imports and aliases. + fn import_path_tokens( + &self, + origin: &IrImportOrigin, + qualifier: &IrImportQualifier, + path: &[String], + ) -> Vec { + let is_pub_library_import = matches!(origin, IrImportOrigin::PubLibrary { .. }); + let is_stdlib = Self::is_incan_source_stdlib_import(origin, qualifier, path); + + if is_stdlib { + let mut tokens = vec![quote! { crate }]; + let std_namespace = Self::rust_ident(stdlib::INCAN_STD_NAMESPACE); + tokens.push(quote! { #std_namespace }); + for seg in path.iter().skip(1) { + let ident = Self::rust_ident(seg); + tokens.push(quote! { #ident }); + } + return tokens; + } + + if is_pub_library_import { + return path + .iter() + .map(|segment| { + let ident = Self::rust_ident(segment); + quote! { #ident } + }) + .collect(); + } + + let mut tokens: Vec = Vec::new(); + match qualifier { + IrImportQualifier::Auto => { + if self.is_internal_module_path(path) { + tokens.push(quote! { crate }); + } + } + IrImportQualifier::Crate => tokens.push(quote! { crate }), + IrImportQualifier::Super(levels) => { + for _ in 0..*levels { + tokens.push(quote! { super }); + } + } + IrImportQualifier::None => {} + } + tokens.extend(path.iter().map(|segment| { + let ident = Self::rust_ident(segment); + quote! { #ident } + })); + tokens + } + + /// Emit the Rust path used by a module-level symbol alias target. + /// + /// Imported targets use their original import path so public aliases re-export public items directly instead of + /// re-exporting a private local `use` binding. + fn emit_symbol_alias_target_path( + &self, + target_origin: Option<&IrImportOrigin>, + target_qualifier: Option<&IrImportQualifier>, + target_path: &[String], + ) -> TokenStream { + let Some(origin) = target_origin else { + let target_segments = target_path + .iter() + .map(|segment| { + let ident = Self::rust_ident(segment); + quote! { #ident } + }) + .collect::>(); + return join_path_tokens(&target_segments); + }; + let Some(qualifier) = target_qualifier else { + let target_segments = target_path + .iter() + .map(|segment| { + let ident = Self::rust_ident(segment); + quote! { #ident } + }) + .collect::>(); + return join_path_tokens(&target_segments); + }; + + let path_tokens = self.import_path_tokens(origin, qualifier, target_path); + let path = join_path_tokens(&path_tokens); + if matches!(qualifier, IrImportQualifier::None) && !matches!(origin, IrImportOrigin::PubLibrary { .. }) { + quote! { :: #path } + } else { + path + } + } + /// Emit a Rust import or re-export after generated-use analysis prunes private unused bindings. fn emit_import( &self, @@ -257,64 +407,9 @@ impl<'a> IrEmitter<'a> { // Only Incan stdlib imports (qualifier `Auto`) are mapped. Rust crate imports like // `from rust::std::collections import HashMap` (qualifier `None`) are left as-is. let is_pub_library_import = matches!(origin, IrImportOrigin::PubLibrary { .. }); - let is_stdlib = - !is_pub_library_import && !matches!(qualifier, IrImportQualifier::None) && stdlib::is_any_stdlib_path(path); - let is_incan_source_stdlib = is_stdlib; + let is_incan_source_stdlib = Self::is_incan_source_stdlib_import(origin, qualifier, path); - let path_tokens: Vec = if is_incan_source_stdlib { - let mut tokens = vec![quote! { crate }]; - let std_namespace = Self::rust_ident(stdlib::INCAN_STD_NAMESPACE); - tokens.push(quote! { #std_namespace }); - for seg in path.iter().skip(1) { - let ident = Self::rust_ident(seg); - tokens.push(quote! { #ident }); - } - tokens - } else if is_pub_library_import { - path.iter() - .map(|segment| { - let ident = Self::rust_ident(segment); - quote! { #ident } - }) - .collect() - } else { - let mut tokens: Vec = Vec::new(); - let mapped_path_tokens: Vec<_> = if is_stdlib { - let mut mapped = vec![quote! { incan_stdlib }]; - // Skip the `std` root, map the rest with keyword escaping. - for seg in path.iter().skip(1) { - let ident = Self::rust_ident(seg); - mapped.push(quote! { #ident }); - } - mapped - } else { - path.iter() - .map(|s| { - let ident = Self::rust_ident(s); - quote! { #ident } - }) - .collect() - }; - let apply_prefix = !is_stdlib; - if apply_prefix { - match qualifier { - IrImportQualifier::Auto => { - if self.is_internal_module_path(path) { - tokens.push(quote! { crate }); - } - } - IrImportQualifier::Crate => tokens.push(quote! { crate }), - IrImportQualifier::Super(levels) => { - for _ in 0..*levels { - tokens.push(quote! { super }); - } - } - IrImportQualifier::None => {} - } - } - tokens.extend(mapped_path_tokens); - tokens - }; + let path_tokens = self.import_path_tokens(origin, qualifier, path); let path_ts = join_path_tokens(&path_tokens); // Public source imports, stdlib facades, and rust.module imports are re-exported. Private `pub::` library @@ -397,12 +492,32 @@ impl<'a> IrEmitter<'a> { && item.name.chars().next().is_some_and(|ch| ch.is_ascii_uppercase())) }) .map(|item| { - let name_ident = Self::rust_ident(&item.name); + let binding = item.alias.as_ref().unwrap_or(&item.name); + let name_ident = if item.is_static { + Self::rust_static_ident(&item.name) + } else { + Self::rust_ident(&item.name) + }; let path_tokens_clone = path_tokens.clone(); let path_ts_clone = join_path_tokens(&path_tokens_clone); let absolute_path = matches!(qualifier, IrImportQualifier::None) && !is_pub_library_import; - if let Some(alias) = &item.alias { - let alias_ident = Self::rust_ident(alias); + let static_init_import = if item.is_static && self.static_needs_imported_init_import(binding) { + let init_ident = Self::rust_ident("__incan_init_module_statics"); + let init_alias = Self::imported_static_init_ident(binding); + if absolute_path { + quote! { use :: #path_ts_clone :: #init_ident as #init_alias; } + } else { + quote! { use #path_ts_clone :: #init_ident as #init_alias; } + } + } else { + quote! {} + }; + let item_import = if let Some(alias) = &item.alias { + let alias_ident = if item.is_static { + Self::rust_static_ident(alias) + } else { + Self::rust_ident(alias) + }; if should_reexport_item(item) { if absolute_path { quote! { pub use :: #path_ts_clone :: #name_ident as #alias_ident; } @@ -430,11 +545,12 @@ impl<'a> IrEmitter<'a> { quote! { use #path_ts_clone :: #name_ident; } } } - } + }; + quote! { #static_init_import #item_import } }) .collect(); Ok(quote! { #(#item_stmts)* }) - } else if path.len() == 1 && !is_stdlib { + } else if path.len() == 1 && !is_incan_source_stdlib { Ok(quote! {}) } else if export_module_import { Ok(quote! { diff --git a/src/backend/ir/emit/decls/mutation_scan.rs b/src/backend/ir/emit/decls/mutation_scan.rs index 8f7911bd8..1d66d42ff 100644 --- a/src/backend/ir/emit/decls/mutation_scan.rs +++ b/src/backend/ir/emit/decls/mutation_scan.rs @@ -316,8 +316,8 @@ impl<'a> IrEmitter<'a> { } IrExprKind::Format { parts } => { for part in parts { - if let super::super::super::expr::FormatPart::Expr(e) = part { - self.scan_expr_for_param_writes(e, param_names, mutated); + if let super::super::super::expr::FormatPart::Expr { expr, .. } = part { + self.scan_expr_for_param_writes(expr, param_names, mutated); } } } diff --git a/src/backend/ir/emit/expressions/calls.rs b/src/backend/ir/emit/expressions/calls.rs index 8d5f1ee01..e499cc056 100644 --- a/src/backend/ir/emit/expressions/calls.rs +++ b/src/backend/ir/emit/expressions/calls.rs @@ -2,15 +2,17 @@ //! //! This module handles emission of regular function calls (user-defined functions) and binary operator expressions. +mod testing_asserts; + use proc_macro2::TokenStream; use quote::quote; -use super::super::super::FunctionSignature; use super::super::super::conversions::{BinOpEmitKind, determine_binop_plan}; use super::super::super::decl::FunctionParam; -use super::super::super::expr::{BinOp, IrCallArg, IrCallArgKind, IrExprKind, TypedExpr, VarAccess, VarRefKind}; -use super::super::super::ownership::{ValueUseSite, incan_call_arg_needs_rust_mut_borrow, plan_value_use}; +use super::super::super::expr::{BinOp, IrCallArg, IrCallArgKind, IrExprKind, TypedExpr, VarRefKind}; +use super::super::super::ownership::{ArgumentPassingPlan, ValueUseSite}; use super::super::super::types::IrType; +use super::super::super::{FunctionRegistry, FunctionSignature}; use super::super::{EmitError, IrEmitter}; use crate::frontend::ast::ParamKind; use incan_core::lang::stdlib; @@ -174,10 +176,30 @@ impl<'a> IrEmitter<'a> { target_ty: &IrType, union_qualifier: Option<&[String]>, ) -> Result, EmitError> { - if arg.ty.is_union() { + self.emit_union_payload_arg_for_site( + arg, + target_ty, + union_qualifier, + ValueUseSite::IncanCallArg { + target_ty: None, + callee_param: None, + in_return: false, + }, + ) + } + + /// Emit a concrete payload argument for a `Union[...]` target while preserving the caller's ownership site. + pub(super) fn emit_union_payload_arg_for_site( + &self, + arg: &TypedExpr, + target_ty: &IrType, + union_qualifier: Option<&[String]>, + site: ValueUseSite<'_>, + ) -> Result, EmitError> { + let Some(value_ty) = self.union_payload_candidate_type(arg, target_ty) else { return Ok(None); - } - let Some(variant_index) = target_ty.union_variant_index_for_member(&arg.ty) else { + }; + let Some(variant_index) = target_ty.union_variant_index_for_member(&value_ty) else { return Ok(None); }; let Some(members) = target_ty.union_members() else { @@ -188,17 +210,35 @@ impl<'a> IrEmitter<'a> { }; let variant_ident = quote::format_ident!("{}", IrType::union_variant_name(variant_index)); let union_path = self.emit_union_type_path_with_qualifier(target_ty, union_qualifier); - let emitted = self.emit_expr_for_use( - arg, - ValueUseSite::IncanCallArg { - target_ty: Some(member_ty), - callee_param: None, - in_return: false, - }, - )?; + let emitted = self.emit_expr_for_use(arg, Self::retarget_value_use_site(site, Some(member_ty)))?; Ok(Some(quote! { #union_path :: #variant_ident(#emitted) })) } + /// Return the concrete union-member payload type for an argument that may already be typed as the target union. + fn union_payload_candidate_type(&self, arg: &TypedExpr, target_ty: &IrType) -> Option { + if !arg.ty.is_union() { + return Some(arg.ty.clone()); + } + + let candidate_name = match &arg.kind { + IrExprKind::Struct { name, .. } => Some(name.as_str()), + IrExprKind::Call { func, .. } => match &func.kind { + IrExprKind::Var { + name, + ref_kind: VarRefKind::TypeName, + .. + } => Some(name.as_str()), + _ => None, + }, + _ => None, + }?; + target_ty + .union_members()? + .iter() + .find(|member| member.nominal_type_name() == Some(candidate_name)) + .cloned() + } + /// Emit a type-seeded literal argument for `None`/`Ok`/`Err` when possible. /// /// This helper rewrites constructor-shaped arguments into explicit generic forms (for example `None::`, `Ok:: IrEmitter<'a> { _ => None, }; let callee_name = local_name.or(canonical_name); - let registry_signature = if canonical_path.is_some() { - canonical_name.and_then(|name| self.function_registry.get(name)) - } else { - local_name - .and_then(|name| self.function_registry.get(name)) - .or_else(|| canonical_name.and_then(|name| self.function_registry.get(name))) - }; - let result_specialized_signature = callable_signature.or(registry_signature).and_then(|signature| { + let merged_signature = FunctionRegistry::effective_call_signature_by( + self.function_registry, + self.canonical_function_registry(), + local_name, + canonical_path, + callable_signature, + Some(&func.ty), + |left, right| self.call_signature_type_matches(left, right), + ); + let result_specialized_signature = merged_signature.as_ref().and_then(|signature| { result_target_ty.and_then(|target_ty| Self::specialize_signature_by_result_target(signature, target_ty)) }); - let function_sig = associated_signature.as_ref().or_else(|| { - if canonical_path.is_some() { - result_specialized_signature - .as_ref() - .or(callable_signature.or(registry_signature)) - } else { - result_specialized_signature - .as_ref() - .or(registry_signature.or(callable_signature)) - } - }); + let function_sig = associated_signature + .as_ref() + .or_else(|| result_specialized_signature.as_ref().or(merged_signature.as_ref())); // The checked-newtype lowering path emits a compiler-internal panic marker call. This remains the narrow, // explicitly-tracked generated `panic!` exemption that issue #351 left to a separate follow-up. Render it as // the Rust `panic!` macro so generated code stays valid without colliding with user-defined functions that may @@ -588,6 +622,7 @@ impl<'a> IrEmitter<'a> { if let Some(sig) = function_sig && sig.params.iter().any(|param| param.kind != ParamKind::Normal) { + let f = Self::call_callee_tokens(func, f, type_args); let arg_tokens = self.emit_rest_aware_call_args(func, args, sig)?; return Ok(quote! { #f #turbofish (#(#arg_tokens),*) }); } @@ -680,18 +715,21 @@ impl<'a> IrEmitter<'a> { }; let target_aware_aggregate_literal_arg = aggregate_literal_arg && !matches!(use_site, ValueUseSite::ExternalCallArg { .. }); + let arg_plan = ArgumentPassingPlan::for_use_site(a, use_site); let previous_qualify = if *from_default { Some(self.qualify_internal_canonical_paths.replace(true)) } else { None }; let emitted = (|| { + let mut emitted_from_seed = false; let emitted = if let Some(target_ty) = target_ty { if let Some(seed) = self.emit_inference_seeded_literal_arg_with_union_qualifier( a, target_ty, pub_library_union_qualifier.as_deref(), )? { + emitted_from_seed = true; seed } else if Self::is_unresolved_call_seed_type(target_ty) { // Signature exists but leaves generics unresolved: fallback to the argument's own inferred @@ -701,6 +739,7 @@ impl<'a> IrEmitter<'a> { &a.ty, pub_library_union_qualifier.as_deref(), )? { + emitted_from_seed = true; seed } else if target_aware_aggregate_literal_arg { self.emit_expr_for_use(a, use_site)? @@ -720,6 +759,7 @@ impl<'a> IrEmitter<'a> { &a.ty, pub_library_union_qualifier.as_deref(), )? { + emitted_from_seed = true; seed } else if target_aware_aggregate_literal_arg { self.emit_expr_for_use(a, use_site)? @@ -727,383 +767,49 @@ impl<'a> IrEmitter<'a> { self.emit_expr(a)? } }; - Ok::(emitted) + Ok::<(TokenStream, bool), EmitError>((emitted, emitted_from_seed)) })(); if let Some(previous) = previous_qualify { self.qualify_internal_canonical_paths.replace(previous); } - let emitted = emitted?; + let (emitted, emitted_from_seed) = emitted?; if let Some(adapter) = self.borrowed_function_adapter_arg(a, target_ty) { return Ok(adapter); } - // Check VarAccess for explicit borrow requirements - if let IrExprKind::Var { access, .. } = &a.kind { - match access { - VarAccess::BorrowMut => return Ok(quote! { &mut #emitted }), - VarAccess::Borrow if matches!(target_ty, Some(IrType::Ref(_) | IrType::RefMut(_)) | None) => { - return Ok(quote! { &#emitted }); - } - _ => {} - } - } - - // Prefer explicit lowering access decisions, then derive obvious borrow requirements from parameter - // typing information. - if let Some(param) = sig_param { - match ¶m.ty { - IrType::Ref(_) => match &a.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &#emitted }), - }, - IrType::RefMut(_) => match &a.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &mut #emitted }), - }, - _ => {} - } - } else if let Some(target_ty) = target_ty { - // Toward #121: when registry metadata is unavailable, use the call expression's function type as a - // borrow hint. - match target_ty { - IrType::RefMut(_) => match &a.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &mut #emitted }), - }, - IrType::Ref(_) => match &a.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &#emitted }), - }, - _ => {} - } - } - - let mut tokens = if target_aware_aggregate_literal_arg { - emitted + let tokens = if emitted_from_seed || target_aware_aggregate_literal_arg { + arg_plan.apply_after_value_plan(emitted) } else { - match use_site { - ValueUseSite::ExternalCallArg { target_ty } => self - .external_list_arg_element_coercion(a, target_ty, emitted.clone()) - .unwrap_or_else(|| plan_value_use(a, use_site).apply(emitted)), - _ => plan_value_use(a, use_site).apply(emitted), - } + arg_plan.apply_full(emitted) }; - if let Some(param) = sig_param - && incan_call_arg_needs_rust_mut_borrow(param) - { - match &a.ty { - IrType::Ref(_) | IrType::RefMut(_) => {} - _ => tokens = quote! { &mut #tokens }, - } - } Ok(tokens) }) .collect::>()?; + let f = Self::call_callee_tokens(func, f, type_args); Ok(quote! { #f #turbofish (#(#arg_tokens),*) }) } - /// Emit canonical RFC 018 assertion helper calls without requiring a source-level `std.testing` import. + /// Parenthesize call targets whose emitted Rust is an expression block rather than a path/call expression. /// - /// Plain `assert` is a language primitive, so its lowered helper calls must remain available even when the - /// explicit stdlib testing module was not imported into the user's source file. - fn try_emit_testing_assert_call( - &self, - canonical_path: Option<&[String]>, - args: &[IrCallArg], - ) -> Result, EmitError> { - let Some(path) = canonical_path else { - return Ok(None); - }; - if path.len() != 3 - || path.first().map(String::as_str) != Some(stdlib::STDLIB_ROOT) - || path.get(1).map(String::as_str) != Some("testing") - { - return Ok(None); - } - let Some(name) = path.last().map(String::as_str) else { - return Ok(None); - }; - - match name { - "assert" => { - let condition = Self::canonical_assert_arg(name, args, 0)?; - let condition_tokens = self.emit_expr(condition)?; - let failure = self.emit_assert_failure("AssertionError", args.get(1).map(|arg| &arg.expr))?; - Ok(Some(quote! { - if !(#condition_tokens) { - #failure - } - })) - } - "assert_false" => { - let condition = Self::canonical_assert_arg(name, args, 0)?; - let condition_tokens = self.emit_expr(condition)?; - let failure = self.emit_assert_failure("AssertionError", args.get(1).map(|arg| &arg.expr))?; - Ok(Some(quote! { - if #condition_tokens { - #failure - } - })) - } - "assert_eq" | "assert_ne" => self.emit_assert_comparison(name, args).map(Some), - "assert_is_some" => self.emit_assert_option_some(args).map(Some), - "assert_is_none" => self.emit_assert_option_none(args).map(Some), - "assert_is_ok" => self.emit_assert_result_ok(args).map(Some), - "assert_is_err" => self.emit_assert_result_err(args).map(Some), - "assert_raises" => self.emit_assert_raises(args).map(Some), - _ => Ok(None), - } - } - - fn canonical_assert_arg<'b>( - helper_name: &str, - args: &'b [IrCallArg], - index: usize, - ) -> Result<&'b TypedExpr, EmitError> { - args.get(index).map(|arg| &arg.expr).ok_or_else(|| { - EmitError::Unsupported(format!( - "canonical std.testing.{helper_name} call missing argument {}", - index + 1 - )) - }) - } - - fn result_constructor_payload(expr: &TypedExpr, constructor: ConstructorId) -> Option<&TypedExpr> { - let expr = match &expr.kind { - IrExprKind::InteropCoerce { expr, .. } => expr.as_ref(), - _ => expr, - }; - if let IrExprKind::Struct { name, fields } = &expr.kind - && name == constructors::as_str(constructor) - { - return fields.first().map(|(_, payload)| payload); - } - let IrExprKind::Call { func, args, .. } = &expr.kind else { - return None; - }; - let IrExprKind::Var { name, .. } = &func.kind else { - return None; - }; - if name != constructors::as_str(constructor) { - return None; - } - args.first().map(|arg| &arg.expr) - } - - fn emit_assert_failure( - &self, - default_message: &'static str, - message: Option<&TypedExpr>, - ) -> Result { - if let Some(message) = message { - let message_tokens = self.emit_expr(message)?; - return Ok(quote! {{ - let __incan_assert_msg = #message_tokens; - if __incan_assert_msg.is_empty() { - panic!(#default_message); - } else { - panic!("AssertionError: {}", __incan_assert_msg); - } - }}); - } - Ok(quote! { panic!(#default_message); }) - } - - fn emit_assert_raises_failure( - &self, - default_message: TokenStream, - message: Option<&TypedExpr>, - ) -> Result { - if let Some(message) = message { - let message_tokens = self.emit_expr(message)?; - return Ok(quote! {{ - let __incan_assert_msg = #message_tokens; - if __incan_assert_msg.is_empty() { - #default_message - } else { - panic!("AssertionError: {}", __incan_assert_msg); - } - }}); - } - Ok(default_message) - } - - fn emit_assert_comparison_failure( - &self, - failure_kind: &'static str, - message: Option<&TypedExpr>, - ) -> Result { - let default_message = format!("AssertionError: {failure_kind}"); - if let Some(message) = message { - let message_tokens = self.emit_expr(message)?; - return Ok(quote! {{ - let __incan_assert_msg = #message_tokens; - if __incan_assert_msg.is_empty() { - panic!(#default_message); - } else { - panic!("AssertionError: {}; {}", __incan_assert_msg, #failure_kind); - } - }}); - } - Ok(quote! { panic!(#default_message); }) - } - - /// Emit canonical `std.testing.assert_eq` / `assert_ne` calls with expression operands isolated. - fn emit_assert_comparison(&self, name: &str, args: &[IrCallArg]) -> Result { - let left = Self::canonical_assert_arg(name, args, 0)?; - let right = Self::canonical_assert_arg(name, args, 1)?; - let left_tokens = self.emit_expr(left)?; - let right_tokens = self.emit_expr(right)?; - let message = args.get(2).map(|arg| &arg.expr); - if name == "assert_eq" { - let failure = self.emit_assert_comparison_failure("left != right", message)?; - Ok(quote! { - if (#left_tokens) != (#right_tokens) { - #failure - } - }) - } else { - let failure = self.emit_assert_comparison_failure("left == right", message)?; - Ok(quote! { - if (#left_tokens) == (#right_tokens) { - #failure - } - }) - } - } - - fn emit_assert_option_some(&self, args: &[IrCallArg]) -> Result { - let option = Self::canonical_assert_arg("assert_is_some", args, 0)?; - let option_tokens = self.emit_expr(option)?; - let failure = self.emit_assert_failure( - "AssertionError: expected Some, got None", - args.get(1).map(|arg| &arg.expr), - )?; - Ok(quote! {{ - let __incan_assert_value = #option_tokens; - match __incan_assert_value { - Some(__incan_assert_inner) => __incan_assert_inner, - None => { - #failure - } - } - }}) - } - - fn emit_assert_option_none(&self, args: &[IrCallArg]) -> Result { - let option = Self::canonical_assert_arg("assert_is_none", args, 0)?; - if matches!(option.kind, IrExprKind::None) { - return Ok(quote! { () }); - } - let option_tokens = self.emit_expr(option)?; - let failure = self.emit_assert_failure( - "AssertionError: expected None, got Some", - args.get(1).map(|arg| &arg.expr), - )?; - Ok(quote! {{ - let __incan_assert_value = #option_tokens; - if __incan_assert_value.is_some() { - #failure - } - }}) - } - - fn emit_assert_result_ok(&self, args: &[IrCallArg]) -> Result { - let result = Self::canonical_assert_arg("assert_is_ok", args, 0)?; - if let Some(payload) = Self::result_constructor_payload(result, ConstructorId::Ok) { - let payload_tokens = Self::emit_result_payload_tokens(payload, self.emit_expr(payload)?); - return Ok(quote! { #payload_tokens }); + /// Storage-rooted method calls materialize arguments and enter `StaticCell::with_ref` / `with_mut`, so their + /// emitted callee has block shape. Calling that result requires `({ ... })(arg)` in Rust. + fn call_callee_tokens(func: &TypedExpr, emitted: TokenStream, type_args: &[IrType]) -> TokenStream { + if !type_args.is_empty() { + return emitted; } - let result_tokens = self.emit_expr(result)?; - let failure = - self.emit_assert_failure("AssertionError: expected Ok, got Err", args.get(1).map(|arg| &arg.expr))?; - Ok(quote! {{ - let __incan_assert_value = #result_tokens; - match __incan_assert_value { - Ok(__incan_assert_inner) => __incan_assert_inner, - Err(_) => { - #failure - } + match &func.kind { + IrExprKind::MethodCall { receiver, .. } if Self::expr_is_storage_rooted(receiver) => { + quote! { ({ #emitted }) } } - }}) - } - - fn emit_assert_result_err(&self, args: &[IrCallArg]) -> Result { - let result = Self::canonical_assert_arg("assert_is_err", args, 0)?; - if let Some(payload) = Self::result_constructor_payload(result, ConstructorId::Err) { - let payload_tokens = Self::emit_result_payload_tokens(payload, self.emit_expr(payload)?); - return Ok(quote! { #payload_tokens }); + IrExprKind::If { .. } + | IrExprKind::Match { .. } + | IrExprKind::Closure { .. } + | IrExprKind::Block { .. } + | IrExprKind::Loop { .. } => quote! { ({ #emitted }) }, + _ => emitted, } - let result_tokens = self.emit_expr(result)?; - let failure = - self.emit_assert_failure("AssertionError: expected Err, got Ok", args.get(1).map(|arg| &arg.expr))?; - Ok(quote! {{ - let __incan_assert_value = #result_tokens; - match __incan_assert_value { - Err(__incan_assert_inner) => __incan_assert_inner, - Ok(_) => { - #failure - } - } - }}) - } - - fn emit_assert_raises(&self, args: &[IrCallArg]) -> Result { - let call = Self::canonical_assert_arg("assert_raises", args, 0)?; - let expected = Self::canonical_assert_arg("assert_raises", args, 1)?; - let call_tokens = self.emit_expr(call)?; - let invocation_tokens = if matches!( - &call.ty, - IrType::Function { params, ret } if params.is_empty() && matches!(ret.as_ref(), IrType::Unit) - ) { - quote! { #call_tokens() } - } else { - quote! { #call_tokens } - }; - let expected_tokens = self.emit_expr(expected)?; - let no_raise = self.emit_assert_raises_failure( - quote! { panic!("AssertionError: expected {} to be raised", __incan_expected_error); }, - args.get(2).map(|arg| &arg.expr), - )?; - let wrong_error = self.emit_assert_raises_failure( - quote! { - panic!( - "AssertionError: expected {} to be raised, got {}", - __incan_expected_error, - __incan_panic_message - ); - }, - args.get(2).map(|arg| &arg.expr), - )?; - - Ok(quote! {{ - let __incan_expected_error = #expected_tokens; - let __incan_raises_result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { - #invocation_tokens; - })); - match __incan_raises_result { - Ok(_) => { - #no_raise - } - Err(__incan_payload) => { - let __incan_panic_message = if let Some(message) = __incan_payload.downcast_ref::() { - message.as_str() - } else if let Some(message) = __incan_payload.downcast_ref::<&str>() { - *message - } else { - "" - }; - let __incan_expected_prefix = format!("{}:", __incan_expected_error); - if __incan_panic_message != __incan_expected_error - && !__incan_panic_message.starts_with(&__incan_expected_prefix) - { - #wrong_error - } - } - } - }}) } pub(in super::super) fn emit_rest_aware_call_args( @@ -1278,54 +984,20 @@ impl<'a> IrEmitter<'a> { in_return, } }; + let arg_plan = ArgumentPassingPlan::for_use_site(arg, use_site); let emitted = if let Some(seed) = self.emit_inference_seeded_literal_arg(arg, ¶m.ty)? { - seed + arg_plan.apply_after_value_plan(seed) } else if Self::is_unresolved_call_seed_type(¶m.ty) { if let Some(seed) = self.emit_inference_seeded_literal_arg(arg, &arg.ty)? { - seed + arg_plan.apply_after_value_plan(seed) } else { - self.emit_expr_for_use(arg, use_site)? + arg_plan.apply_after_value_plan(self.emit_expr_for_use(arg, use_site)?) } } else { - self.emit_expr_for_use(arg, use_site)? - }; - - if let IrExprKind::Var { access, .. } = &arg.kind { - match access { - VarAccess::BorrowMut => return Ok(quote! { &mut #emitted }), - VarAccess::Borrow if matches!(target_ty, Some(IrType::Ref(_) | IrType::RefMut(_)) | None) => { - return Ok(quote! { &#emitted }); - } - _ => {} - } - } - - match ¶m.ty { - IrType::Ref(_) => match &arg.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &#emitted }), - }, - IrType::RefMut(_) => match &arg.ty { - IrType::Ref(_) | IrType::RefMut(_) => return Ok(emitted), - _ => return Ok(quote! { &mut #emitted }), - }, - _ => {} - } - - let mut tokens = match use_site { - ValueUseSite::ExternalCallArg { target_ty } => self - .external_list_arg_element_coercion(arg, target_ty, emitted.clone()) - .unwrap_or(emitted), - _ => emitted, + arg_plan.apply_after_value_plan(self.emit_expr_for_use(arg, use_site)?) }; - if incan_call_arg_needs_rust_mut_borrow(param) { - match &arg.ty { - IrType::Ref(_) | IrType::RefMut(_) => {} - _ => tokens = quote! { &mut #tokens }, - } - } let _ = idx; - Ok(tokens) + Ok(emitted) } /// Emit a canonical callee path when the compiler knows how to materialize that namespace at the current call @@ -1511,7 +1183,7 @@ mod tests { use crate::backend::ir::expr::{ IrCallArg, IrCallArgKind, IrInteropCoercionKind, Literal as IrLiteral, VarAccess, VarRefKind, }; - use crate::backend::ir::types::{IrType, Mutability}; + use crate::backend::ir::types::{IR_UNION_TYPE_NAME, IrType, Mutability}; use crate::backend::ir::{FunctionRegistry, IrEmitter, TypedExpr}; use incan_core::lang::types::numerics::NumericTypeId; @@ -1833,6 +1505,55 @@ mod tests { Ok(()) } + #[test] + fn emit_call_expr_keeps_return_context_union_string_seed_as_union_value() -> Result<(), Box> + { + let union_ty = IrType::NamedGeneric( + IR_UNION_TYPE_NAME.to_string(), + vec![IrType::String, IrType::Bool, IrType::Float, IrType::Int], + ); + let mut registry = FunctionRegistry::new(); + registry.register( + "lit".to_string(), + vec![FunctionParam { + name: "value".to_string(), + ty: union_ty.clone(), + mutability: Mutability::Immutable, + is_self: false, + kind: ParamKind::Normal, + default: None, + }], + IrType::String, + ); + let emitter = IrEmitter::new(®istry); + emitter.in_return_context.replace(true); + let func = TypedExpr::new( + IrExprKind::Var { + name: "lit".to_string(), + access: VarAccess::Copy, + ref_kind: VarRefKind::Value, + }, + IrType::Function { + params: vec![union_ty], + ret: Box::new(IrType::String), + }, + ); + let arg = TypedExpr::new(IrExprKind::String("open".to_string()), IrType::String); + let tokens = emitter + .emit_call_expr(&func, &[], &[pos_arg(arg)], None, None) + .map_err(|err| { + std::io::Error::other(format!( + "union string literal call should emit without post-wrapper coercion: {err:?}" + )) + })?; + + assert_eq!( + render(tokens), + "lit(__IncanUnion43fbd19e99c1db05::V0(\"open\".to_string()))" + ); + Ok(()) + } + #[test] fn emit_call_expr_borrows_struct_arg_for_rust_ref_param() -> Result<(), Box> { let mut registry = FunctionRegistry::new(); @@ -2057,6 +1778,45 @@ mod tests { Ok(()) } + #[test] + fn rest_aware_call_arg_uses_argument_plan_without_double_borrow() -> Result<(), Box> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let func = rust_call_target("takes_ref_rest"); + let signature = FunctionSignature { + params: vec![ + FunctionParam { + name: "value".to_string(), + ty: IrType::Ref(Box::new(IrType::Struct("demo::Thing".to_string()))), + mutability: Mutability::Immutable, + is_self: false, + kind: ParamKind::Normal, + default: None, + }, + FunctionParam { + name: "rest".to_string(), + ty: IrType::List(Box::new(IrType::Int)), + mutability: Mutability::Immutable, + is_self: false, + kind: ParamKind::RestPositional, + default: None, + }, + ], + return_type: IrType::Unit, + }; + let arg = local_arg("value", IrType::Struct("demo::Thing".to_string())); + let tokens = emitter + .emit_call_expr(&func, &[], &[pos_arg(arg)], Some(&signature), None) + .map_err(|err| std::io::Error::other(format!("rest-aware call should emit borrowed arg: {err:?}")))?; + let rendered = render(tokens); + assert!(rendered.starts_with("takes_ref_rest(&value,")); + assert!( + !rendered.contains("&&value"), + "argument plan must not add a second borrow after emit_expr_for_use: {rendered}" + ); + Ok(()) + } + #[test] fn emit_canonical_assert_raises_catches_panic_payloads() -> Result<(), Box> { let registry = FunctionRegistry::new(); diff --git a/src/backend/ir/emit/expressions/calls/testing_asserts.rs b/src/backend/ir/emit/expressions/calls/testing_asserts.rs new file mode 100644 index 000000000..56ea8540e --- /dev/null +++ b/src/backend/ir/emit/expressions/calls/testing_asserts.rs @@ -0,0 +1,349 @@ +use proc_macro2::TokenStream; +use quote::quote; + +use crate::backend::ir::emit::{EmitError, IrEmitter}; +use crate::backend::ir::expr::{IrCallArg, IrExprKind, TypedExpr}; +use crate::backend::ir::types::IrType; +use incan_core::lang::surface::constructors::{self, ConstructorId}; +use incan_core::lang::testing::{self, TestingAssertHelperId}; + +impl<'a> IrEmitter<'a> { + /// Emit canonical RFC 018 assertion helper calls without requiring a source-level `std.testing` import. + /// + /// Plain `assert` is a language primitive, so its lowered helper calls must remain available even when the explicit + /// stdlib testing module was not imported into the user's source file. + pub(super) fn try_emit_testing_assert_call( + &self, + canonical_path: Option<&[String]>, + args: &[IrCallArg], + ) -> Result, EmitError> { + let Some(path) = canonical_path else { + return Ok(None); + }; + let Some(helper_id) = testing::assert_helper_id_from_std_path(path) else { + return Ok(None); + }; + + match helper_id { + TestingAssertHelperId::Assert => { + let condition = Self::canonical_assert_arg(helper_id, args, 0)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(helper_id)?, + args.get(1).map(|arg| &arg.expr), + )?; + if Self::constant_bool(condition) == Some(false) { + return Ok(Some(failure)); + } + let condition_tokens = self.emit_expr(condition)?; + Ok(Some(quote! { + if !(#condition_tokens) { + #failure + } + })) + } + TestingAssertHelperId::AssertFalse => { + let condition = Self::canonical_assert_arg(helper_id, args, 0)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(helper_id)?, + args.get(1).map(|arg| &arg.expr), + )?; + if Self::constant_bool(condition) == Some(true) { + return Ok(Some(failure)); + } + let condition_tokens = self.emit_expr(condition)?; + Ok(Some(quote! { + if #condition_tokens { + #failure + } + })) + } + TestingAssertHelperId::AssertEq | TestingAssertHelperId::AssertNe => { + self.emit_assert_comparison(helper_id, args).map(Some) + } + TestingAssertHelperId::AssertIsSome => self.emit_assert_option_some(args).map(Some), + TestingAssertHelperId::AssertIsNone => self.emit_assert_option_none(args).map(Some), + TestingAssertHelperId::AssertIsOk => self.emit_assert_result_ok(args).map(Some), + TestingAssertHelperId::AssertIsErr => self.emit_assert_result_err(args).map(Some), + TestingAssertHelperId::AssertRaises => self.emit_assert_raises(args).map(Some), + } + } + + fn constant_bool(expr: &TypedExpr) -> Option { + match &expr.kind { + IrExprKind::Bool(value) => Some(*value), + IrExprKind::InteropCoerce { expr, .. } => Self::constant_bool(expr), + _ => None, + } + } + + fn canonical_assert_arg( + helper_id: TestingAssertHelperId, + args: &[IrCallArg], + index: usize, + ) -> Result<&TypedExpr, EmitError> { + let helper_name = testing::assert_helper_as_str(helper_id); + args.get(index).map(|arg| &arg.expr).ok_or_else(|| { + EmitError::Unsupported(format!( + "canonical std.testing.{helper_name} call missing argument {}", + index + 1 + )) + }) + } + + fn assert_failure_message(helper_id: TestingAssertHelperId) -> Result<&'static str, EmitError> { + testing::assert_helper_default_failure_message(helper_id).ok_or_else(|| { + EmitError::Unsupported(format!( + "std.testing.{} does not have a fixed assertion failure message", + testing::assert_helper_as_str(helper_id) + )) + }) + } + + fn result_constructor_payload(expr: &TypedExpr, constructor: ConstructorId) -> Option<&TypedExpr> { + let expr = match &expr.kind { + IrExprKind::InteropCoerce { expr, .. } => expr.as_ref(), + _ => expr, + }; + if let IrExprKind::Struct { name, fields } = &expr.kind + && name == constructors::as_str(constructor) + { + return fields.first().map(|(_, payload)| payload); + } + let IrExprKind::Call { func, args, .. } = &expr.kind else { + return None; + }; + let IrExprKind::Var { name, .. } = &func.kind else { + return None; + }; + if name != constructors::as_str(constructor) { + return None; + } + args.first().map(|arg| &arg.expr) + } + + fn emit_assert_failure( + &self, + default_message: &'static str, + message: Option<&TypedExpr>, + ) -> Result { + if let Some(message) = message { + let message_tokens = self.emit_expr(message)?; + return Ok(quote! {{ + let __incan_assert_msg = #message_tokens; + if __incan_assert_msg.is_empty() { + panic!(#default_message); + } else { + panic!("AssertionError: {}", __incan_assert_msg); + } + }}); + } + Ok(quote! { panic!(#default_message); }) + } + + fn emit_assert_raises_failure( + &self, + default_message: TokenStream, + message: Option<&TypedExpr>, + ) -> Result { + if let Some(message) = message { + let message_tokens = self.emit_expr(message)?; + return Ok(quote! {{ + let __incan_assert_msg = #message_tokens; + if __incan_assert_msg.is_empty() { + #default_message + } else { + panic!("AssertionError: {}", __incan_assert_msg); + } + }}); + } + Ok(default_message) + } + + fn emit_assert_comparison_failure( + &self, + failure_kind: &'static str, + message: Option<&TypedExpr>, + ) -> Result { + let default_message = format!("AssertionError: {failure_kind}"); + if let Some(message) = message { + let message_tokens = self.emit_expr(message)?; + return Ok(quote! {{ + let __incan_assert_msg = #message_tokens; + if __incan_assert_msg.is_empty() { + panic!(#default_message); + } else { + panic!("AssertionError: {}; {}", __incan_assert_msg, #failure_kind); + } + }}); + } + Ok(quote! { panic!(#default_message); }) + } + + /// Emit canonical `std.testing.assert_eq` / `assert_ne` calls with expression operands isolated. + fn emit_assert_comparison( + &self, + helper_id: TestingAssertHelperId, + args: &[IrCallArg], + ) -> Result { + let name = testing::assert_helper_as_str(helper_id); + let left = Self::canonical_assert_arg(helper_id, args, 0)?; + let right = Self::canonical_assert_arg(helper_id, args, 1)?; + let left_tokens = self.emit_expr(left)?; + let right_tokens = self.emit_expr(right)?; + let message = args.get(2).map(|arg| &arg.expr); + let failure_kind = testing::assert_comparison_failure_kind(helper_id).ok_or_else(|| { + EmitError::Unsupported(format!("std.testing.{name} is not a comparison assertion helper")) + })?; + if helper_id == TestingAssertHelperId::AssertEq { + let failure = self.emit_assert_comparison_failure(failure_kind, message)?; + Ok(quote! { + if (#left_tokens) != (#right_tokens) { + #failure + } + }) + } else { + let failure = self.emit_assert_comparison_failure(failure_kind, message)?; + Ok(quote! { + if (#left_tokens) == (#right_tokens) { + #failure + } + }) + } + } + + fn emit_assert_option_some(&self, args: &[IrCallArg]) -> Result { + let option = Self::canonical_assert_arg(TestingAssertHelperId::AssertIsSome, args, 0)?; + let option_tokens = self.emit_expr(option)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(TestingAssertHelperId::AssertIsSome)?, + args.get(1).map(|arg| &arg.expr), + )?; + Ok(quote! {{ + let __incan_assert_value = #option_tokens; + match __incan_assert_value { + Some(__incan_assert_inner) => __incan_assert_inner, + None => { + #failure + } + } + }}) + } + + fn emit_assert_option_none(&self, args: &[IrCallArg]) -> Result { + let option = Self::canonical_assert_arg(TestingAssertHelperId::AssertIsNone, args, 0)?; + if matches!(option.kind, IrExprKind::None) { + return Ok(quote! { () }); + } + let option_tokens = self.emit_expr(option)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(TestingAssertHelperId::AssertIsNone)?, + args.get(1).map(|arg| &arg.expr), + )?; + Ok(quote! {{ + let __incan_assert_value = #option_tokens; + if __incan_assert_value.is_some() { + #failure + } + }}) + } + + fn emit_assert_result_ok(&self, args: &[IrCallArg]) -> Result { + let result = Self::canonical_assert_arg(TestingAssertHelperId::AssertIsOk, args, 0)?; + if let Some(payload) = Self::result_constructor_payload(result, ConstructorId::Ok) { + let payload_tokens = Self::emit_result_payload_tokens(payload, self.emit_expr(payload)?); + return Ok(quote! { #payload_tokens }); + } + let result_tokens = self.emit_expr(result)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(TestingAssertHelperId::AssertIsOk)?, + args.get(1).map(|arg| &arg.expr), + )?; + Ok(quote! {{ + let __incan_assert_value = #result_tokens; + match __incan_assert_value { + Ok(__incan_assert_inner) => __incan_assert_inner, + Err(_) => { + #failure + } + } + }}) + } + + fn emit_assert_result_err(&self, args: &[IrCallArg]) -> Result { + let result = Self::canonical_assert_arg(TestingAssertHelperId::AssertIsErr, args, 0)?; + if let Some(payload) = Self::result_constructor_payload(result, ConstructorId::Err) { + let payload_tokens = Self::emit_result_payload_tokens(payload, self.emit_expr(payload)?); + return Ok(quote! { #payload_tokens }); + } + let result_tokens = self.emit_expr(result)?; + let failure = self.emit_assert_failure( + Self::assert_failure_message(TestingAssertHelperId::AssertIsErr)?, + args.get(1).map(|arg| &arg.expr), + )?; + Ok(quote! {{ + let __incan_assert_value = #result_tokens; + match __incan_assert_value { + Err(__incan_assert_inner) => __incan_assert_inner, + Ok(_) => { + #failure + } + } + }}) + } + + fn emit_assert_raises(&self, args: &[IrCallArg]) -> Result { + let call = Self::canonical_assert_arg(TestingAssertHelperId::AssertRaises, args, 0)?; + let expected = Self::canonical_assert_arg(TestingAssertHelperId::AssertRaises, args, 1)?; + let call_tokens = self.emit_expr(call)?; + let invocation_tokens = if matches!( + &call.ty, + IrType::Function { params, ret } if params.is_empty() && matches!(ret.as_ref(), IrType::Unit) + ) { + quote! { #call_tokens() } + } else { + quote! { #call_tokens } + }; + let expected_tokens = self.emit_expr(expected)?; + let no_raise = self.emit_assert_raises_failure( + quote! { panic!("AssertionError: expected {} to be raised", __incan_expected_error); }, + args.get(2).map(|arg| &arg.expr), + )?; + let wrong_error = self.emit_assert_raises_failure( + quote! { + panic!( + "AssertionError: expected {} to be raised, got {}", + __incan_expected_error, + __incan_panic_message + ); + }, + args.get(2).map(|arg| &arg.expr), + )?; + + Ok(quote! {{ + let __incan_expected_error = #expected_tokens; + let __incan_raises_result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + #invocation_tokens; + })); + match __incan_raises_result { + Ok(_) => { + #no_raise + } + Err(__incan_payload) => { + let __incan_panic_message = if let Some(message) = __incan_payload.downcast_ref::() { + message.as_str() + } else if let Some(message) = __incan_payload.downcast_ref::<&str>() { + *message + } else { + "" + }; + let __incan_expected_prefix = format!("{}:", __incan_expected_error); + if __incan_panic_message != __incan_expected_error + && !__incan_panic_message.starts_with(&__incan_expected_prefix) + { + #wrong_error + } + } + } + }}) + } +} diff --git a/src/backend/ir/emit/expressions/comprehensions.rs b/src/backend/ir/emit/expressions/comprehensions.rs index c5042b96a..34d4bd50c 100644 --- a/src/backend/ir/emit/expressions/comprehensions.rs +++ b/src/backend/ir/emit/expressions/comprehensions.rs @@ -8,11 +8,14 @@ use proc_macro2::TokenStream; use quote::quote; -use super::super::super::expr::{BuiltinFn, IrExprKind, IrGeneratorClause, Pattern, TypedExpr}; +use super::super::super::expr::{ + BuiltinFn, FormatPart, IrCallArg, IrDictEntry, IrExprKind, IrGeneratorClause, IrListEntry, Pattern, TypedExpr, +}; use super::super::super::ownership::{ ComprehensionIterationPlan, dict_comprehension_key_needs_clone, plan_dict_comprehension_iteration, plan_list_comprehension_iteration, plan_owned_iterator_source, }; +use super::super::super::stmt::{AssignTarget, IrStmt, IrStmtKind}; use super::super::super::types::IrType; use super::super::{EmitError, IrEmitter}; @@ -112,20 +115,28 @@ impl<'a> IrEmitter<'a> { // ---- Context: iterator setup ---- let pattern_tokens = self.emit_pattern(pattern); let elem = self.emit_expr(element)?; + let body_can_propagate = Self::expr_contains_try(element) || filter.is_some_and(Self::expr_contains_try); if let Some(iter) = self.emit_direct_comprehension_iterable(iterable)? { + if body_can_propagate { + return self.emit_direct_list_comp_loop(iter, pattern_tokens, elem, filter); + } return self.emit_direct_list_comp(iter, pattern_tokens, elem, filter); } let iter = self.emit_expr(iterable)?; let is_range = self.is_range_iterable(iterable); let iter_wrapped = quote! { (#iter) }; - - match plan_list_comprehension_iteration( + let plan = plan_list_comprehension_iteration( Self::comprehension_iterable_item_ty(&iterable.ty), is_range, filter.is_some(), - ) { + ); + if body_can_propagate { + return self.emit_list_comp_loop(plan, iter_wrapped, pattern, pattern_tokens, elem, filter); + } + + match plan { ComprehensionIterationPlan::RangeFilter => { let Some(filter) = filter else { return Err(EmitError::Unsupported( @@ -211,6 +222,9 @@ impl<'a> IrEmitter<'a> { let pattern_tokens = self.emit_pattern(pattern); let key_tokens = self.emit_expr(key)?; let value_tokens = self.emit_expr(value)?; + let body_can_propagate = Self::expr_contains_try(key) + || Self::expr_contains_try(value) + || filter.is_some_and(Self::expr_contains_try); // ---- Context: key ownership for collected map entries ---- // Dict comprehensions build `(key, value)` tuples left-to-right. For non-Copy keys we clone before the tuple so @@ -224,11 +238,27 @@ impl<'a> IrEmitter<'a> { }; if let Some(iter) = self.emit_direct_comprehension_iterable(iterable)? { + if body_can_propagate { + return self.emit_direct_dict_comp_loop(iter, pattern_tokens, cloned_key, value_tokens, filter); + } return self.emit_direct_dict_comp(iter, pattern_tokens, cloned_key, value_tokens, filter); } let iter = self.emit_expr(iterable)?; - match plan_dict_comprehension_iteration(Self::comprehension_iterable_item_ty(&iterable.ty), filter.is_some()) { + let plan = + plan_dict_comprehension_iteration(Self::comprehension_iterable_item_ty(&iterable.ty), filter.is_some()); + if body_can_propagate { + return self.emit_dict_comp_loop( + plan, + quote! { (#iter) }, + pattern, + pattern_tokens, + (cloned_key, value_tokens), + filter, + ); + } + + match plan { ComprehensionIterationPlan::FilterMapCloneBinding => { let Some(filter) = filter else { return Err(EmitError::Unsupported( @@ -367,6 +397,103 @@ impl<'a> IrEmitter<'a> { } } + /// Emit a direct-iterator list comprehension as an imperative block. + /// + /// This path is used when the element or filter contains `?`. A Rust iterator closure would make `?` target the + /// closure's element-returning type instead of the enclosing Incan function's `Result` return type. + fn emit_direct_list_comp_loop( + &self, + iter: TokenStream, + pattern: TokenStream, + elem: TokenStream, + filter: Option<&TypedExpr>, + ) -> Result { + let body = self.emit_list_comp_push_body(elem, filter)?; + Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #pattern in (#iter) { + #body + } + __incan_list + }}) + } + + /// Emit a planned list comprehension as an imperative block. + fn emit_list_comp_loop( + &self, + plan: ComprehensionIterationPlan, + iter: TokenStream, + pattern: &Pattern, + pattern_tokens: TokenStream, + elem: TokenStream, + filter: Option<&TypedExpr>, + ) -> Result { + let body = self.emit_list_comp_push_body(elem, filter)?; + match plan { + ComprehensionIterationPlan::RangeDirect | ComprehensionIterationPlan::RangeFilter => Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #pattern_tokens in #iter { + #body + } + __incan_list + }}), + ComprehensionIterationPlan::IterCopied => Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #pattern_tokens in #iter.iter().copied() { + #body + } + __incan_list + }}), + ComprehensionIterationPlan::IterCloned => Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #pattern_tokens in #iter.iter().cloned() { + #body + } + __incan_list + }}), + ComprehensionIterationPlan::FilterMapCloneBinding => { + let item_binding = Self::filter_map_item_binding(pattern, &pattern_tokens); + Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #item_binding in #iter.iter() { + let #pattern_tokens = (*#item_binding).clone(); + #body + } + __incan_list + }}) + } + ComprehensionIterationPlan::FilterMapCopyBinding => { + let item_binding = Self::filter_map_item_binding(pattern, &pattern_tokens); + Ok(quote! {{ + let mut __incan_list = Vec::new(); + for #item_binding in #iter.iter() { + let #pattern_tokens = *#item_binding; + #body + } + __incan_list + }}) + } + } + } + + /// Emit one list-comprehension loop body, preserving filter semantics when present. + fn emit_list_comp_push_body( + &self, + elem: TokenStream, + filter: Option<&TypedExpr>, + ) -> Result { + if let Some(filter) = filter { + let filter_tokens = self.emit_expr(filter)?; + Ok(quote! { + if #filter_tokens { + __incan_list.push(#elem); + } + }) + } else { + Ok(quote! { __incan_list.push(#elem); }) + } + } + /// Emit a dict comprehension over an iterable expression that already returns owned values for closure binding. fn emit_direct_dict_comp( &self, @@ -393,4 +520,283 @@ impl<'a> IrEmitter<'a> { Ok(quote! { (#iter).map(|#pattern| (#key, #value)).collect::>() }) } } + + /// Emit a direct-iterator dict comprehension as an imperative block for propagating body expressions. + fn emit_direct_dict_comp_loop( + &self, + iter: TokenStream, + pattern: TokenStream, + key: TokenStream, + value: TokenStream, + filter: Option<&TypedExpr>, + ) -> Result { + let body = self.emit_dict_comp_insert_body(key, value, filter)?; + Ok(quote! {{ + let mut __incan_dict = std::collections::HashMap::new(); + for #pattern in (#iter) { + #body + } + __incan_dict + }}) + } + + /// Emit a planned dict comprehension as an imperative block. + fn emit_dict_comp_loop( + &self, + plan: ComprehensionIterationPlan, + iter: TokenStream, + pattern: &Pattern, + pattern_tokens: TokenStream, + key_value: (TokenStream, TokenStream), + filter: Option<&TypedExpr>, + ) -> Result { + let (key, value) = key_value; + let body = self.emit_dict_comp_insert_body(key, value, filter)?; + match plan { + ComprehensionIterationPlan::IterCopied => Ok(quote! {{ + let mut __incan_dict = std::collections::HashMap::new(); + for #pattern_tokens in #iter.iter().copied() { + #body + } + __incan_dict + }}), + ComprehensionIterationPlan::IterCloned => Ok(quote! {{ + let mut __incan_dict = std::collections::HashMap::new(); + for #pattern_tokens in #iter.iter().cloned() { + #body + } + __incan_dict + }}), + ComprehensionIterationPlan::FilterMapCloneBinding => { + let item_binding = Self::filter_map_item_binding(pattern, &pattern_tokens); + Ok(quote! {{ + let mut __incan_dict = std::collections::HashMap::new(); + for #item_binding in #iter.iter() { + let #pattern_tokens = (*#item_binding).clone(); + #body + } + __incan_dict + }}) + } + ComprehensionIterationPlan::FilterMapCopyBinding => { + let item_binding = Self::filter_map_item_binding(pattern, &pattern_tokens); + Ok(quote! {{ + let mut __incan_dict = std::collections::HashMap::new(); + for #item_binding in #iter.iter() { + let #pattern_tokens = *#item_binding; + #body + } + __incan_dict + }}) + } + ComprehensionIterationPlan::RangeDirect | ComprehensionIterationPlan::RangeFilter => { + unreachable!("dict comprehensions do not use range-specific iteration plans") + } + } + } + + /// Emit one dict-comprehension loop body, preserving filter semantics when present. + fn emit_dict_comp_insert_body( + &self, + key: TokenStream, + value: TokenStream, + filter: Option<&TypedExpr>, + ) -> Result { + if let Some(filter) = filter { + let filter_tokens = self.emit_expr(filter)?; + Ok(quote! { + if #filter_tokens { + __incan_dict.insert(#key, #value); + } + }) + } else { + Ok(quote! { __incan_dict.insert(#key, #value); }) + } + } + + /// Return whether an expression subtree contains `?` and therefore cannot be emitted inside a non-Result Rust + /// iterator closure. + fn expr_contains_try(expr: &TypedExpr) -> bool { + match &expr.kind { + IrExprKind::Try(_) => true, + IrExprKind::BinOp { left, right, .. } => Self::expr_contains_try(left) || Self::expr_contains_try(right), + IrExprKind::UnaryOp { operand, .. } + | IrExprKind::Await(operand) + | IrExprKind::Cast { expr: operand, .. } + | IrExprKind::NumericResize { expr: operand, .. } + | IrExprKind::InteropCoerce { expr: operand, .. } => Self::expr_contains_try(operand), + IrExprKind::Call { func, args, .. } => { + Self::expr_contains_try(func) || args.iter().any(Self::call_arg_contains_try) + } + IrExprKind::BuiltinCall { args, .. } => args.iter().any(Self::expr_contains_try), + IrExprKind::MethodCall { receiver, args, .. } | IrExprKind::KnownMethodCall { receiver, args, .. } => { + Self::expr_contains_try(receiver) || args.iter().any(Self::call_arg_contains_try) + } + IrExprKind::Field { object, .. } => Self::expr_contains_try(object), + IrExprKind::Index { object, index } => Self::expr_contains_try(object) || Self::expr_contains_try(index), + IrExprKind::Slice { + target, + start, + end, + step, + } => { + Self::expr_contains_try(target) + || start.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + || end.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + || step.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::ListComp { + element, + iterable, + filter, + .. + } => { + Self::expr_contains_try(element) + || Self::expr_contains_try(iterable) + || filter.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::DictComp { + key, + value, + iterable, + filter, + .. + } => { + Self::expr_contains_try(key) + || Self::expr_contains_try(value) + || Self::expr_contains_try(iterable) + || filter.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::Generator { element, clauses } => { + Self::expr_contains_try(element) || clauses.iter().any(Self::generator_clause_contains_try) + } + IrExprKind::List(items) => items.iter().any(Self::list_entry_contains_try), + IrExprKind::Dict(entries) => entries.iter().any(Self::dict_entry_contains_try), + IrExprKind::Set(items) | IrExprKind::Tuple(items) => items.iter().any(Self::expr_contains_try), + IrExprKind::Struct { fields, .. } => fields.iter().any(|(_, expr)| Self::expr_contains_try(expr)), + IrExprKind::If { + condition, + then_branch, + else_branch, + } => { + Self::expr_contains_try(condition) + || Self::expr_contains_try(then_branch) + || else_branch.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::Match { scrutinee, arms } => { + Self::expr_contains_try(scrutinee) + || arms.iter().any(|arm| { + arm.guard.as_ref().is_some_and(Self::expr_contains_try) || Self::expr_contains_try(&arm.body) + }) + } + IrExprKind::Closure { body, .. } => Self::expr_contains_try(body), + IrExprKind::Block { stmts, value } => { + stmts.iter().any(Self::stmt_contains_try) + || value.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::Loop { body } => body.iter().any(Self::stmt_contains_try), + IrExprKind::Race { arms, .. } => arms + .iter() + .any(|arm| Self::expr_contains_try(&arm.awaitable) || Self::expr_contains_try(&arm.body)), + IrExprKind::Range { start, end, .. } => { + start.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + || end.as_ref().is_some_and(|expr| Self::expr_contains_try(expr)) + } + IrExprKind::Format { parts } => parts.iter().any(|part| match part { + FormatPart::Literal(_) => false, + FormatPart::Expr { expr, .. } => Self::expr_contains_try(expr), + }), + IrExprKind::Unit + | IrExprKind::None + | IrExprKind::Bool(_) + | IrExprKind::Int(_) + | IrExprKind::IntLiteral(_) + | IrExprKind::Float(_) + | IrExprKind::Decimal(_) + | IrExprKind::String(_) + | IrExprKind::Bytes(_) + | IrExprKind::Var { .. } + | IrExprKind::StaticRead { .. } + | IrExprKind::StaticBinding { .. } + | IrExprKind::AssociatedFunction { .. } + | IrExprKind::Literal(_) + | IrExprKind::FieldsList(_) + | IrExprKind::SerdeToJson + | IrExprKind::SerdeFromJson(_) => false, + } + } + + fn call_arg_contains_try(arg: &IrCallArg) -> bool { + Self::expr_contains_try(&arg.expr) + } + + fn list_entry_contains_try(entry: &IrListEntry) -> bool { + match entry { + IrListEntry::Element(expr) | IrListEntry::Spread(expr) => Self::expr_contains_try(expr), + } + } + + fn dict_entry_contains_try(entry: &IrDictEntry) -> bool { + match entry { + IrDictEntry::Pair(key, value) => Self::expr_contains_try(key) || Self::expr_contains_try(value), + IrDictEntry::Spread(expr) => Self::expr_contains_try(expr), + } + } + + fn generator_clause_contains_try(clause: &IrGeneratorClause) -> bool { + match clause { + IrGeneratorClause::For { iterable, .. } => Self::expr_contains_try(iterable), + IrGeneratorClause::If(condition) => Self::expr_contains_try(condition), + } + } + + fn stmt_contains_try(stmt: &IrStmt) -> bool { + match &stmt.kind { + IrStmtKind::Expr(expr) | IrStmtKind::Let { value: expr, .. } | IrStmtKind::Yield(expr) => { + Self::expr_contains_try(expr) + } + IrStmtKind::Assign { target, value } => { + Self::assign_target_contains_try(target) || Self::expr_contains_try(value) + } + IrStmtKind::CompoundAssign { target, value, .. } => { + Self::assign_target_contains_try(target) || Self::expr_contains_try(value) + } + IrStmtKind::Return(value) | IrStmtKind::Break { value, .. } => { + value.as_ref().is_some_and(Self::expr_contains_try) + } + IrStmtKind::While { condition, body, .. } => { + Self::expr_contains_try(condition) || body.iter().any(Self::stmt_contains_try) + } + IrStmtKind::For { iterable, body, .. } => { + Self::expr_contains_try(iterable) || body.iter().any(Self::stmt_contains_try) + } + IrStmtKind::Loop { body, .. } | IrStmtKind::Block(body) => body.iter().any(Self::stmt_contains_try), + IrStmtKind::If { + condition, + then_branch, + else_branch, + } => { + Self::expr_contains_try(condition) + || then_branch.iter().any(Self::stmt_contains_try) + || else_branch + .as_ref() + .is_some_and(|body| body.iter().any(Self::stmt_contains_try)) + } + IrStmtKind::Match { scrutinee, arms } => { + Self::expr_contains_try(scrutinee) + || arms.iter().any(|arm| { + arm.guard.as_ref().is_some_and(Self::expr_contains_try) || Self::expr_contains_try(&arm.body) + }) + } + IrStmtKind::Continue(_) => false, + } + } + + fn assign_target_contains_try(target: &AssignTarget) -> bool { + match target { + AssignTarget::Field { object, .. } => Self::expr_contains_try(object), + AssignTarget::Index { object, index } => Self::expr_contains_try(object) || Self::expr_contains_try(index), + AssignTarget::Var(_) | AssignTarget::StaticBinding(_) | AssignTarget::Static(_) => false, + } + } } diff --git a/src/backend/ir/emit/expressions/format.rs b/src/backend/ir/emit/expressions/format.rs index d5d936509..6bd49a02d 100644 --- a/src/backend/ir/emit/expressions/format.rs +++ b/src/backend/ir/emit/expressions/format.rs @@ -27,8 +27,8 @@ impl<'a> IrEmitter<'a> { /// ## Notes /// /// - Literal segments are brace-escaped via `incan_core::strings::escape_format_literal`. - /// - Expression segments are formatted via `format!("{}", expr)` before being passed to the semantic-core f-string - /// join helper. + /// - Display expression segments are formatted via `format!("{}", expr)`. + /// - Debug expression segments are formatted via `format!("{:?}", expr)`. pub(in super::super) fn emit_format_expr(&self, parts: &[FormatPart]) -> Result { // Build literal parts (length = args + 1) and a parallel list of formatted args. let mut literal_parts: Vec = Vec::new(); @@ -40,11 +40,15 @@ impl<'a> IrEmitter<'a> { FormatPart::Literal(s) => { current.push_str(&escape_format_literal(s)); } - FormatPart::Expr(e) => { + FormatPart::Expr { expr, style } => { literal_parts.push(current.clone()); current.clear(); - let arg_expr = self.emit_expr(e)?; - args.push(quote! { format!("{}", #arg_expr) }); + let arg_expr = self.emit_expr(expr)?; + if style.emits_rust_debug(&expr.ty) { + args.push(quote! { format!("{:?}", #arg_expr) }); + } else { + args.push(quote! { format!("{}", #arg_expr) }); + } } } } diff --git a/src/backend/ir/emit/expressions/indexing.rs b/src/backend/ir/emit/expressions/indexing.rs index f06cadf5b..d7725d804 100644 --- a/src/backend/ir/emit/expressions/indexing.rs +++ b/src/backend/ir/emit/expressions/indexing.rs @@ -40,27 +40,74 @@ fn emit_dict_lookup_index_key(object: &TypedExpr, index: &TypedExpr, emitted: To } impl<'a> IrEmitter<'a> { - /// Build the fully-qualified generated-module path for a type imported from another emitted module. - /// - /// Default argument expressions can be expanded at a call site outside the module that declared the default. When - /// the default names an enum variant from that declaring module, the generated Rust must qualify the enum type - /// through the dependency module path instead of assuming the type name is locally imported. - fn emit_dependency_type_path(&self, name: &str) -> Option { - if name.contains("::") || self.ambiguous_type_names.contains(name) { - return None; + /// Emit the stable source name for a function-typed value when the value points at a registered generated + /// function. Decorator lowering passes undecorated originals such as `__incan_original_sample`, but source-facing + /// metadata should still report `sample`. + fn emit_callable_name_expr(&self, object: &TypedExpr) -> Result { + let IrType::Function { params, ret } = &object.ty else { + return Ok(quote! { "".to_string() }); + }; + let Some(signature_key) = Self::callable_name_signature_key(params, ret) else { + return Ok(quote! { "".to_string() }); + }; + let callable = self.emit_expr(object)?; + let fn_ty = self.emit_callable_fn_type(params, ret); + + let helper = Self::callable_name_helper_ident(&signature_key); + let mut helper_calls = Vec::new(); + if self.local_callable_name_signature_keys().contains(&signature_key) { + helper_calls.push(quote! { #helper(__incan_callable) }); + } + if let Some(resolution) = self.callable_name_resolutions.get(&signature_key) { + for module_path in &resolution.module_paths { + if module_path == &self.callable_name_current_module_path { + continue; + } + let helper_path = self.emit_callable_name_helper_path(module_path, &signature_key); + helper_calls.push(quote! { #helper_path(__incan_callable) }); + } + } + let fallback = proc_macro2::Literal::string(""); + let mut resolved = quote! { #fallback.to_string() }; + for helper_call in helper_calls.into_iter().rev() { + resolved = quote! { + if let Some(__incan_name) = #helper_call { + __incan_name.to_string() + } else { + #resolved + } + }; + } + + Ok(quote! {{ + let __incan_callable: #fn_ty = #callable; + #resolved + }}) + } + + fn emit_generic_callable_name_expr(&self, object: &TypedExpr) -> Result { + let object = self.emit_expr(object)?; + Ok(quote! { __IncanCallableName::__incan_callable_name(&#object) }) + } + + pub(in crate::backend::ir::emit) fn emit_callable_name_helper_path( + &self, + module_path: &[String], + signature_key: &str, + ) -> TokenStream { + let helper = Self::callable_name_helper_ident(signature_key); + if module_path.is_empty() { + return quote! { crate::#helper }; } - let module_path = self.type_module_paths.get(name)?; let mut segments = vec![quote! { crate }]; for segment in module_path { let ident = Self::rust_ident(segment); segments.push(quote! { #ident }); } - let name_ident = Self::rust_ident(name); - segments.push(quote! { #name_ident }); - + segments.push(quote! { #helper }); let mut iter = segments.into_iter(); - let first = iter.next()?; - Some(iter.fold(first, |acc, segment| quote! { #acc :: #segment })) + let first = iter.next().unwrap_or_else(|| quote! { crate }); + iter.fold(first, |acc, segment| quote! { #acc :: #segment }) } /// Emit an index expression. @@ -218,6 +265,14 @@ impl<'a> IrEmitter<'a> { /// - Tuple field access (`tuple.0` → `tuple.0`) /// - Regular struct field access (`obj.field` → `obj.field`) pub(in super::super) fn emit_field_expr(&self, object: &TypedExpr, field: &str) -> Result { + if field == "__name__" { + return match object.ty { + IrType::Function { .. } => self.emit_callable_name_expr(object), + IrType::Generic(_) => self.emit_generic_callable_name_expr(object), + _ => Ok(quote! { "".to_string() }), + }; + } + if Self::expr_is_storage_rooted(object) { let rewritten = Self::rewrite_storage_root_expr( &TypedExpr::new( diff --git a/src/backend/ir/emit/expressions/interop_coercions.rs b/src/backend/ir/emit/expressions/interop_coercions.rs new file mode 100644 index 000000000..e89e3c5f9 --- /dev/null +++ b/src/backend/ir/emit/expressions/interop_coercions.rs @@ -0,0 +1,156 @@ +use proc_macro2::TokenStream; +use quote::{format_ident, quote}; + +use crate::backend::ir::expr::{IrExprKind, Literal as IrLiteral, TypedExpr}; +use crate::backend::ir::types::IrType; + +/// Emit a typechecker-selected Rust borrow coercion without re-planning ownership at the call site. +pub(super) fn emit_builtin_borrow_coercion( + inner_expr: &TypedExpr, + inner_tokens: TokenStream, + target_ty: &IrType, +) -> TokenStream { + if let Some(emitted) = emit_structural_borrow_coercion(inner_tokens.clone(), target_ty) { + return emitted; + } + match target_ty { + IrType::StrRef => match &inner_expr.ty { + IrType::StaticStr | IrType::StrRef | IrType::FrozenStr | IrType::Ref(_) | IrType::RefMut(_) => { + quote! { #inner_tokens } + } + _ => quote! { &#inner_tokens }, + }, + IrType::Ref(inner) if matches!(inner.as_ref(), IrType::Bytes) => match &inner_expr.ty { + IrType::StaticBytes | IrType::FrozenBytes | IrType::Ref(_) | IrType::RefMut(_) => { + quote! { #inner_tokens } + } + _ => quote! { &#inner_tokens }, + }, + IrType::Ref(inner) | IrType::RefMut(inner) if is_owned_rust_string_target(inner) => { + if expr_already_materializes_owned_string(inner_expr) { + quote! { &#inner_tokens } + } else { + quote! { &(#inner_tokens).to_string() } + } + } + IrType::Ref(inner) | IrType::RefMut(inner) if is_owned_rust_bytes_target(inner) => { + if expr_already_materializes_owned_bytes(inner_expr) { + quote! { &#inner_tokens } + } else { + quote! { &(#inner_tokens).to_vec() } + } + } + IrType::Ref(_) | IrType::RefMut(_) => quote! { &#inner_tokens }, + _ => quote! { #inner_tokens }, + } +} + +/// Return whether an expression already emits an owned Rust `String` value. +fn expr_already_materializes_owned_string(expr: &TypedExpr) -> bool { + matches!(expr.ty, IrType::String) + && !matches!( + expr.kind, + IrExprKind::String(_) | IrExprKind::Literal(IrLiteral::StaticStr(_)) | IrExprKind::StaticRead { .. } + ) +} + +/// Return whether an expression already emits an owned Rust `Vec` value. +fn expr_already_materializes_owned_bytes(expr: &TypedExpr) -> bool { + matches!(expr.ty, IrType::Bytes) && !matches!(expr.kind, IrExprKind::Bytes(_) | IrExprKind::StaticRead { .. }) +} + +/// Return whether a Rust boundary target is an owned Rust string value. +fn is_owned_rust_string_target(ty: &IrType) -> bool { + matches!(ty, IrType::String) + || matches!( + ty, + IrType::Struct(name) if matches!( + name.as_str(), + "String" | "std::string::String" | "alloc::string::String" + ) + ) +} + +/// Return whether a Rust boundary target is an owned Rust byte vector. +fn is_owned_rust_bytes_target(ty: &IrType) -> bool { + matches!(ty, IrType::Bytes) + || matches!( + ty, + IrType::Struct(name) if matches!( + name.as_str(), + "Vec" | "std::vec::Vec" | "alloc::vec::Vec" + ) + ) +} + +/// Emit a projection from a referenced source item into a Rust-boundary target item. +/// +/// Structural borrow coercions iterate source containers so the element expression is usually `&T`. Exact scalar leaves +/// can be copied or cloned from that reference, while borrowed leaves project to the Rust borrow shape the frontend +/// recorded from metadata. +fn emit_structural_borrow_projection(source_tokens: TokenStream, target_ty: &IrType) -> TokenStream { + match target_ty { + IrType::StrRef => quote! { #source_tokens.as_str() }, + IrType::Ref(inner) if matches!(inner.as_ref(), IrType::Bytes) => { + quote! { #source_tokens.as_slice() } + } + IrType::Ref(_) | IrType::RefMut(_) => quote! { #source_tokens }, + IrType::List(inner) => { + let item_ident = format_ident!("__incan_item"); + let item_tokens = emit_structural_borrow_projection(quote! { #item_ident }, inner); + quote! { #source_tokens.iter().map(|#item_ident| #item_tokens).collect::>() } + } + IrType::Set(inner) => { + let item_ident = format_ident!("__incan_item"); + let item_tokens = emit_structural_borrow_projection(quote! { #item_ident }, inner); + quote! { + #source_tokens + .iter() + .map(|#item_ident| #item_tokens) + .collect::>() + } + } + IrType::Dict(key_ty, value_ty) => { + let key_ident = format_ident!("__incan_key"); + let value_ident = format_ident!("__incan_value"); + let key_tokens = emit_structural_borrow_projection(quote! { #key_ident }, key_ty); + let value_tokens = emit_structural_borrow_projection(quote! { #value_ident }, value_ty); + quote! { + #source_tokens + .iter() + .map(|(#key_ident, #value_ident)| (#key_tokens, #value_tokens)) + .collect::>() + } + } + IrType::Option(inner) => { + let item_ident = format_ident!("__incan_item"); + let item_tokens = emit_structural_borrow_projection(quote! { #item_ident }, inner); + quote! { #source_tokens.as_ref().map(|#item_ident| #item_tokens) } + } + IrType::Result(ok_ty, err_ty) => { + let ok_ident = format_ident!("__incan_ok"); + let err_ident = format_ident!("__incan_err"); + let ok_tokens = emit_structural_borrow_projection(quote! { #ok_ident }, ok_ty); + let err_tokens = emit_structural_borrow_projection(quote! { #err_ident }, err_ty); + quote! { + #source_tokens + .as_ref() + .map(|#ok_ident| #ok_tokens) + .map_err(|#err_ident| #err_tokens) + } + } + IrType::Bool | IrType::Int | IrType::Float | IrType::Numeric(_) | IrType::Unit => { + quote! { *#source_tokens } + } + _ => quote! { (*#source_tokens).clone() }, + } +} + +fn emit_structural_borrow_coercion(inner_tokens: TokenStream, target_ty: &IrType) -> Option { + match target_ty { + IrType::List(_) | IrType::Set(_) | IrType::Dict(_, _) | IrType::Option(_) | IrType::Result(_, _) => { + Some(emit_structural_borrow_projection(inner_tokens, target_ty)) + } + _ => None, + } +} diff --git a/src/backend/ir/emit/expressions/methods.rs b/src/backend/ir/emit/expressions/methods.rs index 45d0aec30..c43d7cec1 100644 --- a/src/backend/ir/emit/expressions/methods.rs +++ b/src/backend/ir/emit/expressions/methods.rs @@ -7,14 +7,21 @@ use proc_macro2::TokenStream; use quote::{format_ident, quote}; use super::super::super::FunctionSignature; +use super::super::super::decl::FunctionParam; use super::super::super::expr::{ - CollectionMethodKind, InternalMethodKind, IrCallArg, IrExprKind, IrMethodDispatch, MethodCallArgPolicy, MethodKind, - TypedExpr, VarAccess, VarRefKind, + CollectionMethodKind, InternalMethodKind, IrCallArg, IrCallArgKind, IrExprKind, IrMethodDispatch, + MethodCallArgPolicy, MethodKind, TypedExpr, VarAccess, VarRefKind, }; -use super::super::super::ownership::ValueUseSite; +use super::super::super::ownership::{ + ArgumentPassingPlan, RegularMethodArgumentContext, ValueUseSite, regular_method_argument_use_site, +}; +use super::super::super::reference_shape::{expr_has_rust_reference_shape, type_has_rust_reference_shape}; use super::super::super::types::IrType; use super::super::{EmitError, IrEmitter}; -use incan_core::interop::RustCollectionFamily; +use incan_core::interop::{ + METADATA_FREE_METHOD_BORROW_RULES, MetadataFreeArgClass, MetadataFreeMethodArgBorrowPolicy, + MetadataFreeReceiverClass, RustCollectionFamily, +}; use incan_core::lang::surface::result_methods::{self, ResultMethodId}; mod collection_methods; @@ -31,9 +38,9 @@ use string_methods::emit_string_method; /// /// This deduplicates the pattern of: /// - Detecting `FrozenStr` receivers -/// - Unwrapping them via `.as_str()` +/// - Viewing them through `AsRef` pub(super) struct ReceiverInfo { - /// The receiver token stream (possibly wrapped in `.as_str()` for FrozenStr). + /// The receiver token stream, possibly viewed as `&str` for frozen/imported string values. pub(super) r: TokenStream, /// A borrow of the receiver: `&#r`. pub(super) r_borrow: TokenStream, @@ -44,7 +51,7 @@ impl ReceiverInfo { fn new(receiver_ty: &IrType, emitted: TokenStream) -> Self { let is_frozen_str = matches!(receiver_ty, IrType::FrozenStr); let r = if is_frozen_str { - quote! { #emitted.as_str() } + quote! { <_ as AsRef>::as_ref(&#emitted) } } else { emitted }; @@ -205,6 +212,11 @@ impl<'a> IrEmitter<'a> { }) } } + ResultMethodId::Unwrap | ResultMethodId::UnwrapOr => { + return Err(EmitError::Unsupported(format!( + "Result.{method_name} is not a callback combinator" + ))); + } }; Ok(call) } @@ -227,10 +239,7 @@ impl<'a> IrEmitter<'a> { /// Return whether an argument already has Rust reference shape for a method parameter. fn method_arg_already_borrowed_for_ref_param(arg_ty: &IrType) -> bool { - matches!( - arg_ty, - IrType::Ref(_) | IrType::RefMut(_) | IrType::StrRef | IrType::StaticStr - ) + type_has_rust_reference_shape(arg_ty) } /// Emit method-call arguments with Rust-boundary borrowing and union wrapping applied from callable metadata. @@ -262,17 +271,12 @@ impl<'a> IrEmitter<'a> { let receiver_signature = self .method_signature_for_receiver(&receiver.ty, method) .or(specialized_signature.as_ref()); - let callable_signature = match (callable_signature, receiver_signature) { - (Some(call_sig), Some(method_sig)) - if call_sig.params.iter().all(|param| param.default.is_none()) - && method_sig.params.iter().any(|param| param.default.is_some()) => - { - Some(method_sig) - } - (Some(call_sig), _) => Some(call_sig), - (None, method_sig) => method_sig, - }; - if let Some(sig) = callable_signature + let has_incan_receiver_signature = receiver_signature.is_some(); + let callable_signature = + FunctionSignature::merge_default_source_by(callable_signature, receiver_signature, |left, right| { + self.call_signature_type_matches(left, right) + }); + if let Some(sig) = callable_signature.as_ref() && sig .params .iter() @@ -281,7 +285,7 @@ impl<'a> IrEmitter<'a> { return self.emit_rest_aware_call_args(receiver, args, sig); } - let ordered_args: Vec<(TypedExpr, bool)> = if let Some(sig) = callable_signature { + let ordered_args: Vec<(TypedExpr, bool)> = if let Some(sig) = callable_signature.as_ref() { if args.iter().any(|arg| arg.name.is_some()) { let mut positional: Vec = Vec::new(); let mut named: std::collections::HashMap<&str, TypedExpr> = std::collections::HashMap::new(); @@ -325,12 +329,11 @@ impl<'a> IrEmitter<'a> { .iter() .enumerate() .map(|(idx, (arg, from_default))| { - let param = callable_signature.and_then(|sig| sig.params.get(idx)); + let param = callable_signature.as_ref().and_then(|sig| sig.params.get(idx)); let external_method_shape = matches!( base_use_site, ValueUseSite::ExternalCallArg { .. } | ValueUseSite::MethodArg ); - let external_call_arg_shape = matches!(base_use_site, ValueUseSite::ExternalCallArg { .. }); let arg_use_site = match (base_use_site, param) { (ValueUseSite::ExternalCallArg { .. }, Some(param)) => ValueUseSite::ExternalCallArg { target_ty: Some(¶m.ty), @@ -347,15 +350,27 @@ impl<'a> IrEmitter<'a> { } else { None }; - let external_param_planned = - matches!(arg_use_site, ValueUseSite::ExternalCallArg { target_ty: Some(_) }); let direct_mut_trait_receiver = external_method_shape && idx == 0 && Self::external_trait_first_arg_needs_mut_borrow(receiver, method); + let metadata_free_policy = if (external_method_shape || !has_incan_receiver_signature) + && idx == 0 + && !param.is_some_and(|param| Self::method_arg_already_borrowed_for_ref_param(¶m.ty)) + { + Self::metadata_free_method_arg_borrow_policy(receiver, method, &arg.ty) + } else { + None + }; + let effective_arg_use_site = if metadata_free_policy.is_some() { + ValueUseSite::MethodArg + } else { + arg_use_site + }; + let arg_plan = ArgumentPassingPlan::for_use_site(arg, effective_arg_use_site); let emitted = if direct_mut_trait_receiver { self.emit_expr(arg) } else { - self.emit_expr_for_use(arg, arg_use_site) + self.emit_expr_for_use(arg, effective_arg_use_site) }; if let Some(previous) = previous_qualify { self.qualify_internal_canonical_paths.replace(previous); @@ -388,69 +403,72 @@ impl<'a> IrEmitter<'a> { { return Ok(wrapped); } + if let Some(policy) = metadata_free_policy { + emitted = match policy { + MetadataFreeMethodArgBorrowPolicy::Shared if !expr_has_rust_reference_shape(arg) => { + quote! { &#emitted } + } + MetadataFreeMethodArgBorrowPolicy::Mutable => quote! { &mut #emitted }, + MetadataFreeMethodArgBorrowPolicy::Shared => emitted, + }; + } let Some(param) = param else { - if external_method_shape && idx == 0 && Self::method_arg_needs_fallback_mut_borrow(method, &arg.ty) - { - emitted = quote! { &mut #emitted }; - } else if external_method_shape - && idx == 0 - && Self::method_arg_needs_fallback_borrow(method, &arg.ty) - { - emitted = quote! { &#emitted }; - } return Ok(emitted); }; if let Some(wrapped) = self.emit_union_payload_arg(arg, ¶m.ty, None)? { - return Ok(wrapped); - } - if external_call_arg_shape - && let Some(coerced) = - self.external_list_arg_element_coercion(arg, Some(¶m.ty), emitted.clone()) - { - emitted = coerced; - } - if external_method_shape - && !external_param_planned - && idx == 0 - && Self::method_arg_needs_fallback_mut_borrow(method, &arg.ty) - { - return Ok(quote! { &mut #emitted }); - } - if external_method_shape - && !external_param_planned - && idx == 0 - && Self::method_arg_needs_fallback_borrow(method, &arg.ty) - { - return Ok(quote! { &#emitted }); - } - if !external_param_planned { - match ¶m.ty { - IrType::Ref(_) if matches!(base_use_site, ValueUseSite::MethodArg) => {} - IrType::Ref(_) => match &arg.ty { - _ if Self::method_arg_already_borrowed_for_ref_param(&arg.ty) => {} - _ => emitted = quote! { &#emitted }, - }, - IrType::RefMut(_) => match &arg.ty { - IrType::Ref(_) | IrType::RefMut(_) => {} - _ => emitted = quote! { &mut #emitted }, - }, - _ => {} - } + return Ok(arg_plan.apply_after_value_plan(wrapped)); } - Ok(emitted) + Ok(arg_plan.apply_after_value_plan(emitted)) }) .collect() } - /// Return whether an external Rust method's first argument should be emitted as a mutable borrow. - fn method_arg_needs_fallback_mut_borrow(method: &str, arg_ty: &IrType) -> bool { - match method { - "read_to_string" => true, - "read" | "read_to_end" | "read_exact" | "read_buf" | "read_buf_exact" => Self::is_byte_buffer_type(arg_ty), - _ => false, + /// Return the explicitly registered compatibility borrow policy for a metadata-free external method argument. + /// + /// Signature metadata remains the source of truth for Rust-boundary borrowing. These policies are only for + /// default-build interop surfaces emitted without rust-inspect metadata. + fn metadata_free_method_arg_borrow_policy( + receiver: &TypedExpr, + method: &str, + arg_ty: &IrType, + ) -> Option { + METADATA_FREE_METHOD_BORROW_RULES.iter().find_map(|rule| { + if !rule.methods.contains(&method) { + return None; + } + if !Self::metadata_free_receiver_matches(receiver, rule.receiver) { + return None; + } + if !Self::metadata_free_arg_matches(arg_ty, rule.arg) { + return None; + } + Some(rule.policy) + }) + } + + fn metadata_free_receiver_matches(receiver: &TypedExpr, class: MetadataFreeReceiverClass) -> bool { + match class { + MetadataFreeReceiverClass::IoValue => Self::receiver_allows_io_method_fallback(receiver), + MetadataFreeReceiverClass::EncodingInstance => { + Self::receiver_type_matches_any(receiver, &["Encoding", "encoding_rs::Encoding"]) + } + MetadataFreeReceiverClass::ExternalAssociated => Self::is_external_associated_receiver(receiver), + } + } + + fn metadata_free_arg_matches(arg_ty: &IrType, class: MetadataFreeArgClass) -> bool { + match class { + MetadataFreeArgClass::StringBuffer => Self::is_string_buffer_type(arg_ty), + MetadataFreeArgClass::ByteBuffer => Self::is_byte_buffer_type(arg_ty), + MetadataFreeArgClass::Any => true, } } + /// Return whether a metadata-free receiver is eligible for std::io-style compatibility borrowing. + fn receiver_allows_io_method_fallback(receiver: &TypedExpr) -> bool { + !Self::expr_is_type_like(receiver) + } + /// Return whether an external Rust trait-style associated call needs `&mut` for its first argument. fn external_trait_first_arg_needs_mut_borrow(receiver: &TypedExpr, method: &str) -> bool { if !matches!(method, "update" | "finalize_xof_reset") { @@ -466,24 +484,55 @@ impl<'a> IrEmitter<'a> { ) } - /// Return whether an external Rust method's first argument should be emitted as a shared borrow. - fn method_arg_needs_fallback_borrow(method: &str, arg_ty: &IrType) -> bool { - match method { - "write_all" => true, - "for_label" | "decode" | "encode" => true, - "write" => Self::is_byte_buffer_type(arg_ty), - _ => false, - } + /// Return whether a metadata-free method receiver is an external Rust associated-call target. + fn is_external_associated_receiver(receiver: &TypedExpr) -> bool { + matches!( + &receiver.kind, + IrExprKind::Var { + ref_kind: VarRefKind::ExternalRustName, + .. + } + ) && Self::expr_is_type_like(receiver) + } + + /// Return whether the receiver's nominal type name matches one of the expected Rust compatibility surfaces. + fn receiver_type_matches_any(receiver: &TypedExpr, expected: &[&str]) -> bool { + Self::receiver_type_for_method_dispatch(&receiver.ty) + .nominal_type_name() + .is_some_and(|name| { + let short_name = name.rsplit("::").next().unwrap_or(name); + expected.iter().any(|expected_name| { + name == *expected_name || short_name == expected_name.rsplit("::").next().unwrap_or(expected_name) + }) + }) } /// Return whether an IR type can stand in for a mutable Rust byte buffer. fn is_byte_buffer_type(ty: &IrType) -> bool { matches!(ty, IrType::Bytes | IrType::FrozenBytes) + || matches!( + ty, + IrType::Struct(name) + if matches!(name.as_str(), "Vec" | "std::vec::Vec" | "alloc::vec::Vec") + ) || matches!( ty, IrType::NamedGeneric(name, args) - if matches!(name.as_str(), "Vec" | "std::vec::Vec") - && matches!(args.as_slice(), [IrType::Int]) + if matches!(name.as_str(), "Vec" | "std::vec::Vec" | "alloc::vec::Vec") + && matches!( + args.as_slice(), + [IrType::Int | IrType::Numeric(incan_core::lang::types::numerics::NumericTypeId::U8)] + ) + ) + } + + /// Return whether an IR type can stand in for a mutable Rust string buffer. + fn is_string_buffer_type(ty: &IrType) -> bool { + matches!(ty, IrType::String) + || matches!( + ty, + IrType::Struct(name) + if matches!(name.as_str(), "String" | "std::string::String" | "alloc::string::String") ) } @@ -506,24 +555,38 @@ impl<'a> IrEmitter<'a> { /// Materialize method-call arguments before entering a static storage lock. /// /// This prevents lock reentry when argument expressions also read/write static-backed values. - fn materialize_storage_rooted_args( + fn materialize_storage_rooted_args<'site>( &self, args: &[IrCallArg], + callable_signature: Option<&'site FunctionSignature>, + base_use_site: ValueUseSite<'site>, ) -> Result<(Vec, Vec), EmitError> { let mut bindings = Vec::with_capacity(args.len()); let mut rewritten = Vec::with_capacity(args.len()); for (idx, arg) in args.iter().enumerate() { let name = format!("__incan_static_arg_{idx}"); let ident = format_ident!("{}", name); - let emitted = self.emit_expr(&arg.expr)?; - bindings.push(quote! { let #ident = #emitted; }); + let param = Self::signature_param_for_original_call_arg(args, idx, callable_signature); + let materialize_site = Self::storage_arg_materialization_use_site(base_use_site, param); + let emitted = self.emit_expr_for_use(&arg.expr, materialize_site)?; + let mutable = + param.is_some_and(|param| matches!(param.mutability, super::super::super::types::Mutability::Mutable)); + let binding = if mutable { + quote! { let mut #ident = #emitted; } + } else { + quote! { let #ident = #emitted; } + }; + bindings.push(binding); + let rewritten_ty = param + .map(|param| param.ty.clone()) + .unwrap_or_else(|| arg.expr.ty.clone()); let rewritten_expr = TypedExpr::new( IrExprKind::Var { name, - access: VarAccess::Read, + access: VarAccess::Move, ref_kind: VarRefKind::Value, }, - arg.expr.ty.clone(), + rewritten_ty, ) .with_ownership(arg.expr.ownership) .with_span(arg.expr.span); @@ -536,6 +599,56 @@ impl<'a> IrEmitter<'a> { Ok((bindings, rewritten)) } + /// Combine pre-lock argument materialization with the storage access expression as one Rust expression block. + fn storage_rooted_method_expr(arg_bindings: Vec, wrapped: TokenStream) -> TokenStream { + quote! {{ + #(#arg_bindings)* + #wrapped + }} + } + + /// Return the callable parameter matched by one original call argument before storage-lock materialization. + fn signature_param_for_original_call_arg<'sig>( + args: &[IrCallArg], + idx: usize, + callable_signature: Option<&'sig FunctionSignature>, + ) -> Option<&'sig FunctionParam> { + let signature = callable_signature?; + let arg = args.get(idx)?; + if matches!(arg.kind, IrCallArgKind::PositionalUnpack | IrCallArgKind::KeywordUnpack) { + return None; + } + if let Some(name) = arg.name.as_deref() { + return signature.params.iter().find(|param| param.name == name); + } + let positional_idx = args + .iter() + .take(idx) + .filter(|arg| arg.name.is_none() && matches!(arg.kind, IrCallArgKind::Positional)) + .count(); + signature.params.get(positional_idx) + } + + /// Pick the use-site plan used when evaluating one storage-rooted method argument before taking the storage lock. + fn storage_arg_materialization_use_site<'site>( + base_use_site: ValueUseSite<'site>, + param: Option<&'site FunctionParam>, + ) -> ValueUseSite<'site> { + match (base_use_site, param) { + (ValueUseSite::IncanCallArg { in_return, .. }, Some(param)) => ValueUseSite::IncanCallArg { + target_ty: Some(¶m.ty), + callee_param: Some(param), + in_return, + }, + (ValueUseSite::ExternalCallArg { .. }, Some(param)) | (ValueUseSite::MethodArg, Some(param)) => { + ValueUseSite::ExternalCallArg { + target_ty: Some(¶m.ty), + } + } + (site, _) => site, + } + } + /// Strip reference wrappers from a receiver type before builtin-family or ownership-sensitive dispatch. /// /// Method emission cares about the underlying receiver family (`Dict`, `Struct`, `Trait`, ...) rather than whether @@ -575,6 +688,17 @@ impl<'a> IrEmitter<'a> { } } + /// Return whether a receiver is a zero-cost `rusttype` alias over an external Rust type. + fn is_rusttype_alias_receiver(&self, receiver_ty: &IrType) -> bool { + match Self::receiver_type_for_method_dispatch(receiver_ty) { + IrType::Struct(name) | IrType::NamedGeneric(name, _) => { + let short_name = name.rsplit("::").next().unwrap_or(name); + self.rusttype_alias_names.contains(name) || self.rusttype_alias_names.contains(short_name) + } + _ => false, + } + } + /// Recover a field receiver's declared surface type before choosing method-call ownership policy. fn receiver_with_known_field_type(&self, receiver: &TypedExpr) -> Option { let IrExprKind::Field { object, field } = &receiver.kind else { @@ -617,30 +741,29 @@ impl<'a> IrEmitter<'a> { args: &[IrCallArg], ) -> Result { if Self::expr_is_storage_rooted(receiver) { - let (arg_bindings, rewritten_args) = self.materialize_storage_rooted_args(args)?; + let (arg_bindings, rewritten_args) = + self.materialize_storage_rooted_args(args, None, ValueUseSite::MethodArg)?; if matches!(kind, MethodKind::Collection(CollectionMethodKind::Get)) { let rewritten_receiver = Self::rewrite_storage_root_expr(receiver, "__incan_static_value"); let arg_exprs: Vec = rewritten_args.iter().map(|a| a.expr.clone()).collect(); let inner = self.emit_static_collection_get(&rewritten_receiver, &arg_exprs)?; let wrapped = self.emit_storage_with_ref(receiver, inner)?; - return Ok(quote! { - #(#arg_bindings)* - #wrapped - }); + return Ok(Self::storage_rooted_method_expr(arg_bindings, wrapped)); } - let rewritten_receiver = Self::rewrite_storage_root_expr(receiver, "__incan_static_value"); - let inner = self.emit_known_method_call(&rewritten_receiver, kind, &rewritten_args)?; let use_mut = super::method_kind_uses_mutable_receiver(kind); + let rewritten_receiver = if use_mut { + Self::rewrite_storage_root_expr_for_mut(receiver, "__incan_static_value") + } else { + Self::rewrite_storage_root_expr(receiver, "__incan_static_value") + }; + let inner = self.emit_known_method_call(&rewritten_receiver, kind, &rewritten_args)?; let wrapped = if use_mut { self.emit_storage_with_mut(receiver, inner) } else { self.emit_storage_with_ref(receiver, inner) }?; - return Ok(quote! { - #(#arg_bindings)* - #wrapped - }); + return Ok(Self::storage_rooted_method_expr(arg_bindings, wrapped)); } let r0 = self.emit_expr(receiver)?; @@ -650,6 +773,28 @@ impl<'a> IrEmitter<'a> { MethodKind::String(kind) => emit_string_method(self, &info, kind, &arg_exprs), MethodKind::Collection(kind) => emit_collection_method(self, receiver, &info, kind, &arg_exprs), MethodKind::Iterator(kind) => emit_iterator_method(self, receiver, &info, kind, &arg_exprs), + MethodKind::Result(ResultMethodId::Unwrap) => { + if !arg_exprs.is_empty() { + return Err(EmitError::Unsupported("Result.unwrap expects no arguments".to_string())); + } + let receiver_tokens = &info.r; + Ok(quote! { + match #receiver_tokens { + Ok(__incan_ok) => __incan_ok, + Err(_) => panic!("called Result.unwrap() on an Err value"), + } + }) + } + MethodKind::Result(ResultMethodId::UnwrapOr) => { + let Some(default) = arg_exprs.first() else { + return Err(EmitError::Unsupported( + "Result.unwrap_or expects one default argument".to_string(), + )); + }; + let default_tokens = self.emit_expr(default)?; + let receiver_tokens = &info.r; + Ok(quote! { #receiver_tokens.unwrap_or(#default_tokens) }) + } MethodKind::Result(kind) => { let Some(callback) = arg_exprs.first() else { return Err(EmitError::Unsupported(format!( @@ -728,8 +873,38 @@ impl<'a> IrEmitter<'a> { result_use_site: Option>, ) -> Result { if Self::expr_is_storage_rooted(receiver) { - let (arg_bindings, rewritten_args) = self.materialize_storage_rooted_args(args)?; - let rewritten_receiver = Self::rewrite_storage_root_expr(receiver, "__incan_static_value"); + let use_mut = !matches!(arg_policy, MethodCallArgPolicy::PreserveShape); + let rewritten_receiver = if use_mut { + Self::rewrite_storage_root_expr_for_mut(receiver, "__incan_static_value") + } else { + Self::rewrite_storage_root_expr(receiver, "__incan_static_value") + }; + let in_return = *self.in_return_context.borrow(); + let receiver_ref_kind = match &rewritten_receiver.kind { + IrExprKind::Var { ref_kind, .. } => Some(*ref_kind), + _ => None, + }; + let has_incan_method_signature = self + .method_signature_for_receiver(&rewritten_receiver.ty, method) + .is_some(); + let preserve_lookup_arg_shape = matches!(arg_policy, MethodCallArgPolicy::PreserveShape) + || rust_collection_family_for_ir_type(&rewritten_receiver.ty) + .is_some_and(|family| family.preserves_lookup_arg_shape(method)); + let rusttype_alias_receiver = self.is_rusttype_alias_receiver(&rewritten_receiver.ty); + let base_use_site = regular_method_argument_use_site( + RegularMethodArgumentContext { + arg_policy, + receiver_ref_kind, + has_incan_method_signature, + is_incan_owned_nominal_receiver: self.is_incan_owned_nominal_receiver(&rewritten_receiver.ty), + is_rusttype_alias_receiver: rusttype_alias_receiver, + preserves_lookup_arg_shape: preserve_lookup_arg_shape, + in_return, + }, + None, + ); + let (arg_bindings, rewritten_args) = + self.materialize_storage_rooted_args(args, callable_signature, base_use_site)?; let inner = self.emit_method_call_expr_with_result_use( &rewritten_receiver, method, @@ -740,15 +915,12 @@ impl<'a> IrEmitter<'a> { arg_policy, result_use_site, )?; - let wrapped = if matches!(arg_policy, MethodCallArgPolicy::PreserveShape) { - self.emit_storage_with_ref(receiver, inner) - } else { + let wrapped = if use_mut { self.emit_storage_with_mut(receiver, inner) + } else { + self.emit_storage_with_ref(receiver, inner) }?; - return Ok(quote! { - #(#arg_bindings)* - #wrapped - }); + return Ok(Self::storage_rooted_method_expr(arg_bindings, wrapped)); } let inferred_receiver = self.receiver_with_known_field_type(receiver); @@ -899,29 +1071,19 @@ impl<'a> IrEmitter<'a> { let preserve_lookup_arg_shape = matches!(arg_policy, MethodCallArgPolicy::PreserveShape) || rust_collection_family_for_ir_type(&receiver.ty) .is_some_and(|family| family.preserves_lookup_arg_shape(method)); - let use_site = if receiver_ref_kind != Some(VarRefKind::ExternalRustName) - && (has_incan_method_signature || self.is_incan_owned_nominal_receiver(&receiver.ty)) - { - ValueUseSite::IncanCallArg { - target_ty: None, - callee_param: None, - in_return: false, - } - } else if receiver_ref_kind == Some(VarRefKind::ExternalName) { - // Module-qualified calls like `widgets.make_widget(...)` are function namespace lookups, not external Rust - // methods. They should keep ordinary Incan/public-function conversions instead of Rust interop coercions. - ValueUseSite::IncanCallArg { - target_ty: None, - callee_param: None, + let rusttype_alias_receiver = self.is_rusttype_alias_receiver(&receiver.ty); + let use_site = regular_method_argument_use_site( + RegularMethodArgumentContext { + arg_policy, + receiver_ref_kind, + has_incan_method_signature, + is_incan_owned_nominal_receiver: self.is_incan_owned_nominal_receiver(&receiver.ty), + is_rusttype_alias_receiver: rusttype_alias_receiver, + preserves_lookup_arg_shape: preserve_lookup_arg_shape, in_return, - } - } else if preserve_lookup_arg_shape { - // Borrow-sensitive collection lookups must keep the source argument shape instead of applying - // function-style coercions such as `.to_string()` / `.into()`. - ValueUseSite::MethodArg - } else { - ValueUseSite::ExternalCallArg { target_ty: None } - }; + }, + None, + ); let arg_tokens = self.emit_method_call_args(method, receiver, args, callable_signature, use_site, result_target_ty)?; Ok(quote! { #r.#m #method_turbofish (#(#arg_tokens),*) }) diff --git a/src/backend/ir/emit/expressions/methods/fast_paths.rs b/src/backend/ir/emit/expressions/methods/fast_paths.rs index 879cf0341..c2ffa2228 100644 --- a/src/backend/ir/emit/expressions/methods/fast_paths.rs +++ b/src/backend/ir/emit/expressions/methods/fast_paths.rs @@ -152,10 +152,10 @@ fn is_owned_string_type(ty: &IrType) -> bool { fn borrowed_str_tokens(ty: &IrType, emitted: TokenStream) -> TokenStream { match ty { IrType::StaticStr | IrType::StrRef => emitted, - IrType::FrozenStr => quote! { #emitted.as_str() }, + IrType::FrozenStr => quote! { <_ as AsRef>::as_ref(&#emitted) }, IrType::Ref(inner) | IrType::RefMut(inner) => match peel_refs(inner) { IrType::StaticStr | IrType::StrRef => emitted, - IrType::FrozenStr => quote! { #emitted.as_str() }, + IrType::FrozenStr => quote! { <_ as AsRef>::as_ref(#emitted) }, _ => quote! { <_ as AsRef>::as_ref(#emitted) }, }, _ => quote! { <_ as AsRef>::as_ref(&#emitted) }, diff --git a/src/backend/ir/emit/expressions/mod.rs b/src/backend/ir/emit/expressions/mod.rs index 4972c6f85..4e1c30976 100644 --- a/src/backend/ir/emit/expressions/mod.rs +++ b/src/backend/ir/emit/expressions/mod.rs @@ -45,6 +45,7 @@ mod calls; mod comprehensions; mod format; mod indexing; +mod interop_coercions; mod lvalue; mod methods; mod structs_enums; @@ -59,7 +60,7 @@ use super::super::expr::{ }; use super::super::types::IrType; use super::{EmitError, IrEmitter}; -use crate::backend::ir::ownership::{ValueUseSite, plan_value_use}; +use crate::backend::ir::ownership::{ValueUseSite, plan_value_use, value_use_site_target_ty}; use incan_core::lang::types::collections::{self, CollectionTypeId}; #[derive(Debug, Clone)] @@ -91,31 +92,6 @@ pub(in crate::backend::ir::emit) fn method_kind_uses_mutable_receiver(kind: &Met } impl<'a> IrEmitter<'a> { - /// Convert a direct `Vec` argument into `Vec` at external Rust call boundaries. - /// - /// The Incan typechecker does not prove Rust `From` relationships. At an external Rust boundary, Rust's own - /// trait checker is the source of truth, so this emits an element-level `.into()` map only when metadata says the - /// parameter expects a different direct list element type. - pub(super) fn external_list_arg_element_coercion( - &self, - arg: &TypedExpr, - target_ty: Option<&IrType>, - emitted: TokenStream, - ) -> Option { - let Some(IrType::List(target_elem)) = target_ty else { - return None; - }; - let IrType::List(source_elem) = &arg.ty else { - return None; - }; - if source_elem == target_elem || Self::is_unresolved_call_seed_type(target_elem) { - return None; - } - Some(quote! { - (#emitted).into_iter().map(|__incan_item| ::std::convert::Into::into(__incan_item)).collect::>() - }) - } - /// Build a typed tuple-field read for compiler-expanded tuple unpacking. pub(super) fn tuple_field_expr(expr: &TypedExpr, idx: usize, ty: IrType) -> TypedExpr { TypedExpr::new( @@ -273,16 +249,7 @@ impl<'a> IrEmitter<'a> { /// Return the target type carried by a value-use site, if the site has one. fn use_site_target_ty<'b>(site: ValueUseSite<'b>) -> Option<&'b IrType> { - match site { - ValueUseSite::IncanCallArg { target_ty, .. } - | ValueUseSite::ExternalCallArg { target_ty } - | ValueUseSite::StructField { target_ty } - | ValueUseSite::CollectionElement { target_ty } - | ValueUseSite::Assignment { target_ty } - | ValueUseSite::ReturnValue { target_ty } - | ValueUseSite::MatchScrutinee { target_ty } => target_ty, - ValueUseSite::MethodArg => None, - } + value_use_site_target_ty(site) } /// Prefer the call-site target type for aggregate literal elements. @@ -350,11 +317,16 @@ impl<'a> IrEmitter<'a> { /// expression is emitted. Non-aggregate expressions are emitted normally, then the planned conversion is applied to /// the resulting token stream. pub(super) fn emit_expr_for_use(&self, expr: &TypedExpr, site: ValueUseSite<'_>) -> Result { - if matches!(site, ValueUseSite::CollectionElement { .. }) - && let Some(target_ty) = Self::use_site_target_ty(site) - && let Some(wrapped) = self.emit_inference_seeded_literal_arg(expr, target_ty)? - { - return Ok(wrapped); + let resolved_target_ty = Self::use_site_target_ty(site).map(|ty| self.resolve_type_aliases_for_emit(ty)); + if let Some(target_ty) = resolved_target_ty.as_ref() { + if let Some(wrapped) = self.emit_union_payload_arg_for_site(expr, target_ty, None, site)? { + return Ok(wrapped); + } + if matches!(site, ValueUseSite::CollectionElement { .. }) + && let Some(wrapped) = self.emit_inference_seeded_literal_arg(expr, target_ty)? + { + return Ok(wrapped); + } } match &expr.kind { @@ -374,7 +346,7 @@ impl<'a> IrEmitter<'a> { return self.emit_expr_for_use(inner, site); } IrExprKind::List(items) => { - let site_item_ty = match Self::use_site_target_ty(site) { + let site_item_ty = match resolved_target_ty.as_ref() { Some(IrType::List(elem)) => Some(elem.as_ref()), _ => None, }; @@ -386,7 +358,7 @@ impl<'a> IrEmitter<'a> { return self.emit_list_literal_entries(items, item_target_ty); } IrExprKind::Dict(pairs) => { - let (site_key_ty, site_value_ty) = match Self::use_site_target_ty(site) { + let (site_key_ty, site_value_ty) = match resolved_target_ty.as_ref() { Some(IrType::Dict(key, value)) => (Some(key.as_ref()), Some(value.as_ref())), _ => (None, None), }; @@ -402,7 +374,7 @@ impl<'a> IrEmitter<'a> { if items.is_empty() { return Ok(quote! { std::collections::HashSet::new() }); } - let site_item_ty = match Self::use_site_target_ty(site) { + let site_item_ty = match resolved_target_ty.as_ref() { Some(IrType::Set(elem)) => Some(elem.as_ref()), _ => None, }; @@ -425,7 +397,7 @@ impl<'a> IrEmitter<'a> { return Ok(quote! { [#(#item_tokens),*].into_iter().collect::>() }); } IrExprKind::Tuple(items) => { - let site_tuple_items = match Self::use_site_target_ty(site) { + let site_tuple_items = match resolved_target_ty.as_ref() { Some(IrType::Tuple(items)) => Some(items.as_slice()), _ => None, }; @@ -483,13 +455,18 @@ impl<'a> IrEmitter<'a> { callable_signature, canonical_path, } => { + let target_site = if let Some(target_ty) = resolved_target_ty.as_ref() { + Self::retarget_value_use_site(site, Some(target_ty)) + } else { + site + }; return self.emit_call_expr_for_use( func, type_args, args, callable_signature.as_ref(), canonical_path.as_deref(), - site, + target_site, ); } _ => {} @@ -555,15 +532,31 @@ impl<'a> IrEmitter<'a> { Self::expr_storage_root(expr).is_some() } + /// Rewrite a static/storage binding root to the local borrowed value used inside `with_ref`. pub(super) fn rewrite_storage_root_expr(expr: &TypedExpr, local_name: &str) -> TypedExpr { + Self::rewrite_storage_root_expr_inner(expr, local_name, false) + } + + /// Rewrite a static/storage binding root to the local mutable borrow used inside `with_mut`. + pub(super) fn rewrite_storage_root_expr_for_mut(expr: &TypedExpr, local_name: &str) -> TypedExpr { + Self::rewrite_storage_root_expr_inner(expr, local_name, true) + } + + /// Rewrite the root of a storage-backed path while preserving the original field/index chain. + fn rewrite_storage_root_expr_inner(expr: &TypedExpr, local_name: &str, mutable_root: bool) -> TypedExpr { let replacement = || { + let ty = if mutable_root { + IrType::RefMut(Box::new(expr.ty.clone())) + } else { + expr.ty.clone() + }; TypedExpr::new( IrExprKind::Var { name: local_name.to_string(), access: super::super::expr::VarAccess::Read, ref_kind: VarRefKind::Value, }, - expr.ty.clone(), + ty, ) }; @@ -575,14 +568,14 @@ impl<'a> IrEmitter<'a> { } => replacement(), IrExprKind::Field { object, field } => TypedExpr::new( IrExprKind::Field { - object: Box::new(Self::rewrite_storage_root_expr(object, local_name)), + object: Box::new(Self::rewrite_storage_root_expr_inner(object, local_name, mutable_root)), field: field.clone(), }, expr.ty.clone(), ), IrExprKind::Index { object, index } => TypedExpr::new( IrExprKind::Index { - object: Box::new(Self::rewrite_storage_root_expr(object, local_name)), + object: Box::new(Self::rewrite_storage_root_expr_inner(object, local_name, mutable_root)), index: index.clone(), }, expr.ty.clone(), @@ -599,7 +592,7 @@ impl<'a> IrEmitter<'a> { match Self::expr_storage_root(expr) { Some(StorageRoot::Static(name)) => { let ident = Self::rust_static_ident(&name); - let init_call = self.emit_module_static_init_call(); + let init_call = self.emit_static_init_call_for_static(&name); Ok(quote! {{ #init_call #ident.with_ref(|#local_name| { #body }) @@ -618,7 +611,7 @@ impl<'a> IrEmitter<'a> { match Self::expr_storage_root(expr) { Some(StorageRoot::Static(name)) => { let ident = Self::rust_static_ident(&name); - let init_call = self.emit_module_static_init_call(); + let init_call = self.emit_static_init_call_for_static(&name); Ok(quote! {{ #init_call #ident.with_mut(|#local_name| { #body }) @@ -695,16 +688,21 @@ impl<'a> IrEmitter<'a> { Ok(quote! { #n.get() }) } IrExprKind::Var { name, access: _, .. } => { + if *self.qualify_internal_canonical_paths.borrow() + && let Some(path) = self.emit_dependency_value_path(name) + { + return Ok(path); + } let n = Self::rust_ident(name); Ok(quote! { #n }) } IrExprKind::StaticRead { name } => { let n = Self::rust_static_ident(name); - if *self.in_static_initializer.borrow() { + if *self.in_static_initializer.borrow() && !self.static_needs_imported_init_call(name) { Ok(quote! { #n.get() }) } else { - let init_call = self.emit_module_static_init_call(); + let init_call = self.emit_static_init_call_for_static(name); Ok(quote! {{ #init_call #n.get() @@ -714,10 +712,10 @@ impl<'a> IrEmitter<'a> { IrExprKind::StaticBinding { name } => { let n = Self::rust_static_ident(name); - if *self.in_static_initializer.borrow() { + if *self.in_static_initializer.borrow() && !self.static_needs_imported_init_call(name) { Ok(quote! { incan_stdlib::storage::StaticBinding::from_static(&#n) }) } else { - let init_call = self.emit_module_static_init_call(); + let init_call = self.emit_static_init_call_for_static(name); Ok(quote! {{ #init_call incan_stdlib::storage::StaticBinding::from_static(&#n) @@ -1017,58 +1015,47 @@ impl<'a> IrEmitter<'a> { IrExprKind::InteropCoerce { expr: inner, from_ty: _, - to_ty: _, + to_ty, kind, } => { - let inner = self.emit_expr(inner)?; + let inner_tokens = self.emit_expr(inner)?; match kind { IrInteropCoercionKind::Builtin { policy, rust_target } => { - let rust_target = rust_target.replace(' ', ""); let emitted = match policy { - incan_core::interop::CoercionPolicy::Exact => match rust_target.as_str() { - "String" | "std::string::String" => { - quote! { (#inner).to_string() } - } - "Vec" | "std::vec::Vec" => { - quote! { (#inner).to_vec() } - } - _ => quote! { #inner }, + incan_core::interop::CoercionPolicy::Exact => match to_ty { + IrType::String => quote! { (#inner_tokens).to_string() }, + IrType::Bytes => quote! { (#inner_tokens).to_vec() }, + _ => quote! { #inner_tokens }, }, incan_core::interop::CoercionPolicy::Lossless => { - let target = syn::parse_str::(rust_target.as_str()).map_err(|err| { + let target = self.emit_type(to_ty); + let _: syn::Type = syn::parse2(target.clone()).map_err(|err| { EmitError::SynParse(format!( "invalid Rust boundary cast target `{rust_target}`: {err}" )) })?; - quote! { (#inner) as #target } + quote! { (#inner_tokens) as #target } + } + incan_core::interop::CoercionPolicy::Borrow => { + interop_coercions::emit_builtin_borrow_coercion(inner, inner_tokens, to_ty) } - incan_core::interop::CoercionPolicy::Borrow => match rust_target.as_str() { - "&str" | "&[u8]" => quote! { &#inner }, - "&String" | "&std::string::String" | "&alloc::string::String" => { - quote! { &(#inner).to_string() } - } - "&Vec" | "&std::vec::Vec" | "&alloc::vec::Vec" => { - quote! { &(#inner).to_vec() } - } - _ => quote! { &#inner }, - }, incan_core::interop::CoercionPolicy::Lossy => match rust_target.as_str() { - "f32" => quote! { (#inner) as f32 }, - _ => quote! { #inner }, + "f32" => quote! { (#inner_tokens) as f32 }, + _ => quote! { #inner_tokens }, }, }; Ok(emitted) } IrInteropCoercionKind::AdapterCall { adapter, adapter_kind } => { let adapter = self.emit_expr(adapter)?; - let call = quote! { #adapter(#inner) }; + let call = quote! { #adapter(#inner_tokens) }; let emitted = match adapter_kind { IrInteropAdapterKind::Via => call, IrInteropAdapterKind::Try => quote! { #call? }, }; Ok(emitted) } - IrInteropCoercionKind::RustTypeUnwrap => Ok(quote! { #inner }), + IrInteropCoercionKind::RustTypeUnwrap => Ok(quote! { #inner_tokens }), } } @@ -1104,6 +1091,20 @@ mod tests { use crate::backend::ir::{FunctionParam, FunctionRegistry, FunctionSignature, Mutability}; use incan_core::lang::traits::{self as core_traits, TraitId}; + fn prost_decode_signature(return_type: IrType) -> FunctionSignature { + FunctionSignature { + params: vec![FunctionParam { + name: "buf".to_string(), + ty: IrType::Generic("Buf".to_string()), + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }], + return_type, + } + } + #[test] fn type_name_associated_call_does_not_borrow_string_arguments() -> Result<(), String> { let registry = FunctionRegistry::new(); @@ -1294,6 +1295,355 @@ mod tests { Ok(()) } + #[test] + fn external_decode_metadata_keeps_explicit_slice_argument_shape() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let descriptor_set = IrType::Struct("prost_types::FileDescriptorSet".to_string()); + let result_ty = IrType::Result( + Box::new(descriptor_set.clone()), + Box::new(IrType::Struct("prost::DecodeError".to_string())), + ); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "FileDescriptorSet".to_string(), + access: VarAccess::Copy, + ref_kind: VarRefKind::ExternalRustName, + }, + descriptor_set.clone(), + )), + method: "decode".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "data".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Bytes, + )), + method: "as_slice".to_string(), + dispatch: None, + type_args: Vec::new(), + args: Vec::new(), + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Bytes, + ), + }], + callable_signature: Some(prost_decode_signature(result_ty.clone())), + arg_policy: MethodCallArgPolicy::Default, + }, + result_ty, + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("FileDescriptorSet :: decode (data . as_slice ())"), + "explicit slice arguments should be passed through, got `{rendered}`" + ); + assert!( + !rendered.contains("FileDescriptorSet :: decode (& data . as_slice ())"), + "decode metadata must not add a fallback borrow to explicit slice arguments, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn external_decode_metadata_keeps_explicit_rust_vec_slice_argument_shape() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let descriptor_set = IrType::Struct("prost_types::FileDescriptorSet".to_string()); + let result_ty = IrType::Result( + Box::new(descriptor_set.clone()), + Box::new(IrType::Struct("prost::DecodeError".to_string())), + ); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "FileDescriptorSet".to_string(), + access: VarAccess::Copy, + ref_kind: VarRefKind::ExternalRustName, + }, + descriptor_set.clone(), + )), + method: "decode".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "encoded".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("alloc::vec::Vec".to_string()), + )), + method: "as_slice".to_string(), + dispatch: None, + type_args: Vec::new(), + args: Vec::new(), + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Bytes, + ), + }], + callable_signature: Some(prost_decode_signature(result_ty.clone())), + arg_policy: MethodCallArgPolicy::Default, + }, + result_ty, + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("FileDescriptorSet :: decode (encoded . as_slice ())"), + "explicit Rust Vec slice arguments should be passed through, got `{rendered}`" + ); + assert!( + !rendered.contains("FileDescriptorSet :: decode (& encoded . as_slice ())"), + "decode metadata must not add a fallback borrow to explicit Rust Vec slice arguments, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn external_decode_fallback_still_borrows_owned_bytes_argument() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let descriptor_set = IrType::Struct("prost_types::FileDescriptorSet".to_string()); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "FileDescriptorSet".to_string(), + access: VarAccess::Copy, + ref_kind: VarRefKind::ExternalRustName, + }, + descriptor_set.clone(), + )), + method: "decode".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "data".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Bytes, + ), + }], + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Result( + Box::new(descriptor_set), + Box::new(IrType::Struct("prost::DecodeError".to_string())), + ), + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("FileDescriptorSet :: decode (& data)"), + "owned bytes should still use the decode fallback borrow, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn encoding_decode_compatibility_policy_overrides_incomplete_by_value_signature() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "enc".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("encoding_rs::Encoding".to_string()), + )), + method: "decode".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "data".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Bytes, + ), + }], + callable_signature: Some(FunctionSignature { + params: vec![FunctionParam { + name: "bytes".to_string(), + ty: IrType::Bytes, + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }], + return_type: IrType::Unknown, + }), + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Unknown, + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("enc . decode (& data)"), + "encoding_rs decode should borrow bytes even when the recovered signature is incomplete, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn unregistered_decode_method_with_by_value_metadata_preserves_argument_shape() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "decoder".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("ExternalDecoder".to_string()), + )), + method: "decode".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "data".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Bytes, + ), + }], + callable_signature: Some(FunctionSignature { + params: vec![FunctionParam { + name: "data".to_string(), + ty: IrType::Bytes, + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }], + return_type: IrType::Unknown, + }), + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Unknown, + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("decoder . decode (data)"), + "explicit by-value metadata must preserve argument shape, got `{rendered}`" + ); + assert!( + !rendered.contains("decoder . decode (& data)") && !rendered.contains("decoder.decode(&data)"), + "explicit by-value metadata must not use the metadata-free byte borrow default, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn metadata_free_read_to_string_fallback_requires_string_buffer() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "reader".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("ExternalReader".to_string()), + )), + method: "read_to_string".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "count".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Int, + ), + }], + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Unknown, + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("reader . read_to_string (count)"), + "read_to_string fallback should preserve non-string argument shape, got `{rendered}`" + ); + assert!( + !rendered.contains("reader . read_to_string (& mut count)") + && !rendered.contains("reader.read_to_string(&mut count)"), + "read_to_string fallback must not mutably borrow non-string arguments, got `{rendered}`" + ); + Ok(()) + } + #[test] fn interop_try_adapter_emits_question_mark() -> Result<(), String> { let registry = FunctionRegistry::new(); @@ -1347,13 +1697,13 @@ mod tests { IrType::String, )), from_ty: IrType::String, - to_ty: IrType::Ref(Box::new(IrType::String)), + to_ty: IrType::Ref(Box::new(IrType::Struct("String".to_string()))), kind: IrInteropCoercionKind::Builtin { policy: incan_core::interop::CoercionPolicy::Borrow, - rust_target: "&String".to_string(), + rust_target: "&str".to_string(), }, }, - IrType::Ref(Box::new(IrType::String)), + IrType::Ref(Box::new(IrType::Struct("String".to_string()))), ); let emitted = emitter @@ -1371,6 +1721,88 @@ mod tests { Ok(()) } + #[test] + fn interop_borrowed_string_coercion_borrows_owned_string_without_materializing() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::InteropCoerce { + expr: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "text".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::String, + )), + from_ty: IrType::String, + to_ty: IrType::Ref(Box::new(IrType::Struct("String".to_string()))), + kind: IrInteropCoercionKind::Builtin { + policy: incan_core::interop::CoercionPolicy::Borrow, + rust_target: "&str".to_string(), + }, + }, + IrType::Ref(Box::new(IrType::Struct("String".to_string()))), + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered == "& text" || rendered == "&text", + "expected borrowed owned String interop coercion to borrow directly, got `{rendered}`" + ); + assert!( + !rendered.contains("to_string"), + "owned String borrow coercions must not clone through `.to_string()`, got `{rendered}`" + ); + Ok(()) + } + + #[test] + fn interop_structural_list_borrow_coercion_projects_str_items() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::InteropCoerce { + expr: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "items".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::List(Box::new(IrType::String)), + )), + from_ty: IrType::List(Box::new(IrType::String)), + to_ty: IrType::List(Box::new(IrType::StrRef)), + kind: IrInteropCoercionKind::Builtin { + policy: incan_core::interop::CoercionPolicy::Borrow, + rust_target: "Vec<&str>".to_string(), + }, + }, + IrType::List(Box::new(IrType::StrRef)), + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains("items . iter ()"), + "expected structural borrow coercion to iterate source list, got `{rendered}`" + ); + assert!( + rendered.contains("as_str ()"), + "expected structural borrow coercion to project string items as &str, got `{rendered}`" + ); + assert!( + rendered.contains("collect :: < Vec < _ >> ()"), + "expected structural borrow coercion to collect a Rust Vec, got `{rendered}`" + ); + Ok(()) + } + #[test] fn interop_wrapped_dict_literal_keeps_call_site_value_target() -> Result<(), String> { let registry = FunctionRegistry::new(); @@ -1501,6 +1933,38 @@ mod tests { Ok(()) } + #[test] + fn interop_borrowed_vec_bytes_coercion_materializes_owned_bytes_before_borrow() -> Result<(), String> { + let registry = FunctionRegistry::new(); + let emitter = IrEmitter::new(®istry); + let expr = TypedExpr::new( + IrExprKind::InteropCoerce { + expr: Box::new(TypedExpr::new(IrExprKind::Bytes(b"abc".to_vec()), IrType::StaticBytes)), + from_ty: IrType::StaticBytes, + to_ty: IrType::Ref(Box::new(IrType::Struct("Vec".to_string()))), + kind: IrInteropCoercionKind::Builtin { + policy: incan_core::interop::CoercionPolicy::Borrow, + rust_target: "&[u8]".to_string(), + }, + }, + IrType::Ref(Box::new(IrType::Struct("Vec".to_string()))), + ); + + let emitted = emitter + .emit_expr(&expr) + .map_err(|err| format!("expected successful expression emission, got {err:?}"))?; + let rendered = emitted.to_string(); + assert!( + rendered.contains(". to_vec ()"), + "expected borrowed Vec interop coercion to materialize owned bytes, got `{rendered}`" + ); + assert!( + rendered.starts_with("&"), + "expected borrowed Vec interop coercion to emit a borrow, got `{rendered}`" + ); + Ok(()) + } + #[test] fn non_string_method_call_join_stays_regular_method_call() -> Result<(), String> { let registry = FunctionRegistry::new(); @@ -2306,7 +2770,7 @@ mod tests { } #[test] - fn qualified_rusttype_receiver_method_uses_incan_string_conversion() -> Result<(), String> { + fn qualified_rusttype_receiver_method_uses_rust_signature_borrowing() -> Result<(), String> { let registry = FunctionRegistry::new(); let mut emitter = IrEmitter::new(®istry); emitter.rusttype_alias_names.insert("_RawRegex".to_string()); @@ -2345,7 +2809,17 @@ mod tests { IrType::String, ), }], - callable_signature: None, + callable_signature: Some(FunctionSignature { + params: vec![FunctionParam { + name: "text".to_string(), + ty: IrType::Ref(Box::new(IrType::String)), + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }], + return_type: IrType::Struct("_RawMatchIterator".to_string()), + }), arg_policy: MethodCallArgPolicy::Default, }, IrType::Struct("_RawMatchIterator".to_string()), @@ -2360,8 +2834,12 @@ mod tests { "expected regular method-call emission on qualified rusttype receiver, got `{rendered}`" ); assert!( - !rendered.contains("& text") && !rendered.contains("&text"), - "qualified rusttype receiver methods must use Incan arg rules for owned string args, got `{rendered}`" + rendered.contains("find_iter (& text)") || rendered.contains("find_iter (&text)"), + "metadata-resolved rusttype receiver methods should borrow owned strings for Rust &str params, got `{rendered}`" + ); + assert!( + !rendered.contains("to_string"), + "metadata-resolved rusttype receiver methods should not clone strings before borrowing, got `{rendered}`" ); Ok(()) } diff --git a/src/backend/ir/emit/mod.rs b/src/backend/ir/emit/mod.rs index 23a578415..9d47cde1b 100644 --- a/src/backend/ir/emit/mod.rs +++ b/src/backend/ir/emit/mod.rs @@ -31,7 +31,7 @@ use std::cell::RefCell; use std::collections::{HashMap, HashSet}; use proc_macro2::TokenStream; -use quote::quote; +use quote::{format_ident, quote}; use super::decl::{IrDeclKind, IrEnumValue, IrEnumValueType, IrStruct, VariantFields, Visibility}; use super::expr::TypedExpr; @@ -67,6 +67,22 @@ pub(crate) struct ExternalOrdinalCustomKey { pub has_ordinal_bytes_equal: bool, } +/// Cross-module callable-name resolver metadata keyed by a concrete function-pointer signature. +#[derive(Debug, Clone)] +pub(crate) struct CallableNameResolution { + pub(super) params: Vec, + pub(super) ret: IrType, + pub(super) module_paths: Vec>, +} + +/// Callable-name usage facts collected from one lowered program. +#[derive(Debug, Clone, Default)] +pub(crate) struct CallableNameUseFacts { + pub(crate) signature_keys: HashSet, + pub(crate) function_arg_signature_keys: HashSet, + pub(crate) generic_trait_used: bool, +} + /// Usage facts collected before Rust emission. /// /// This analysis is intentionally about generated Rust lints, not source-language reachability diagnostics. It records @@ -94,6 +110,31 @@ pub(super) struct GeneratedUseAnalysis { pub(super) result_observer_callable_types: HashSet, /// Top-level function values adapted to a borrowed function-pointer parameter. pub(super) borrowed_function_adapters: HashSet<(String, Vec)>, + /// Concrete function-pointer signatures whose values read `__name__`. + pub(super) callable_name_signature_keys: HashSet, + /// Concrete top-level function signatures passed through reachable calls. + pub(super) callable_name_function_arg_signature_keys: HashSet, + /// Whether a generic callable parameter reads `__name__` through the generated callable-name trait. + pub(super) uses_generic_callable_name_trait: bool, +} + +impl GeneratedUseAnalysis { + /// Return whether generated Rust should retain an impl method under the current program-level preservation mode. + pub(super) fn should_retain_method( + &self, + preserve_public_items: bool, + target_type: &str, + method_name: &str, + visibility: &Visibility, + ) -> bool { + self.public_types.contains(target_type) + || (!preserve_public_items + && !matches!(visibility, Visibility::Private) + && self.reachable_items.contains(target_type)) + || self + .used_methods + .contains(&(target_type.to_string(), method_name.to_string())) + } } #[derive(Clone)] @@ -194,8 +235,10 @@ pub struct IrEmitter<'a> { emit_zen_in_main: bool, /// Whether serde is needed for emitted Rust derives or helpers. needs_serde: RefCell, - /// Function registry for call-site type checking + /// Function registry for module-local call-site default argument filling and type-aware argument conversion. function_registry: &'a FunctionRegistry, + /// Cross-module registry used only for IR calls that carry an explicit canonical callee path. + canonical_function_registry: Option, /// Track struct derives for generating serde methods in impl blocks struct_derives: std::collections::HashMap>, /// Current function's return type (for applying conversions in return statements) @@ -220,6 +263,8 @@ pub struct IrEmitter<'a> { struct_field_defaults: std::collections::HashMap<(String, String), super::IrExpr>, /// Constructor metadata variants for source-defined structs that share a simple name across modules. struct_constructor_metadata: HashMap>, + /// Transparent local type aliases keyed by alias name. + type_aliases: HashMap, /// Incan `rusttype` aliases that should use compiler-owned call conversion rules at the surface boundary. rusttype_alias_names: HashSet, /// Method signature lookup for Incan-owned nominal receivers, including imported modules. @@ -234,6 +279,10 @@ pub struct IrEmitter<'a> { type_module_paths: HashMap>, /// Type names that are declared in multiple modules (ambiguous). ambiguous_type_names: HashSet, + /// Map of value name -> module path segments for dependency modules. + value_module_paths: HashMap>, + /// Value names that are declared in multiple modules (ambiguous). + ambiguous_value_names: HashSet, /// Imported enum type names discovered from dependency modules. /// /// Imported enums usually lower to `IrType::Struct(name)` in consumer modules, so for-loop emission needs this @@ -265,6 +314,11 @@ pub struct IrEmitter<'a> { newtype_checked_ctor: HashMap, /// Whether the currently emitted module contains any local `static` declarations. module_has_local_statics: RefCell, + /// Imported static bindings that need their defining module's static-init guard before use. + imported_static_init_bindings: RefCell>, + /// Imported static bindings re-exported by this module whose defining module's static-init guard should be + /// chained from this module's init helper. + imported_static_module_init_bindings: RefCell>, /// Whether expression emission is currently inside a static initializer. /// /// Used to avoid recursively forcing the module-wide static init helper while generating static initializer code. @@ -296,6 +350,15 @@ pub struct IrEmitter<'a> { emitted_result_observer_callable_helpers: RefCell>, /// Top-level function values adapted to a borrowed function-pointer parameter. borrowed_function_adapters: RefCell)>>, + /// Current generated Rust module path. The crate root uses an empty path. + callable_name_current_module_path: Vec, + /// Concrete callable-name helper modules available to this compilation unit. + callable_name_resolutions: HashMap, + /// Concrete callable-name signatures used somewhere in this compilation unit. + callable_name_used_signature_keys: HashSet, + /// Local callable registry used for module-local callable-name helpers when the main emitter has a unified + /// cross-module call registry. + callable_name_local_registry: Option, } impl<'a> IrEmitter<'a> { @@ -314,6 +377,7 @@ impl<'a> IrEmitter<'a> { emit_zen_in_main: false, needs_serde: RefCell::new(false), function_registry, + canonical_function_registry: None, struct_derives: std::collections::HashMap::new(), current_function_return_type: RefCell::new(None), external_rust_functions: std::collections::HashSet::new(), @@ -326,6 +390,7 @@ impl<'a> IrEmitter<'a> { struct_field_descriptions: std::collections::HashMap::new(), struct_field_defaults: std::collections::HashMap::new(), struct_constructor_metadata: HashMap::new(), + type_aliases: HashMap::new(), rusttype_alias_names: HashSet::new(), method_signatures: HashMap::new(), method_signature_type_params: HashMap::new(), @@ -333,6 +398,8 @@ impl<'a> IrEmitter<'a> { const_string_literals: std::collections::HashMap::new(), type_module_paths: HashMap::new(), ambiguous_type_names: HashSet::new(), + value_module_paths: HashMap::new(), + ambiguous_value_names: HashSet::new(), dependency_enum_types: HashSet::new(), external_error_trait_types: HashSet::new(), internal_module_roots: HashSet::new(), @@ -340,6 +407,8 @@ impl<'a> IrEmitter<'a> { rust_import_paths: RefCell::new(std::collections::HashMap::new()), newtype_checked_ctor: HashMap::new(), module_has_local_statics: RefCell::new(false), + imported_static_init_bindings: RefCell::new(HashSet::new()), + imported_static_module_init_bindings: RefCell::new(Vec::new()), in_static_initializer: RefCell::new(false), qualify_internal_canonical_paths: RefCell::new(false), qualify_union_types_from_crate: false, @@ -349,11 +418,223 @@ impl<'a> IrEmitter<'a> { result_observer_callable_types: RefCell::new(HashSet::new()), emitted_result_observer_callable_helpers: RefCell::new(HashSet::new()), borrowed_function_adapters: RefCell::new(HashSet::new()), + callable_name_current_module_path: Vec::new(), + callable_name_resolutions: HashMap::new(), + callable_name_used_signature_keys: HashSet::new(), + callable_name_local_registry: None, + } + } + + /// Configure the generated Rust module path for callable-name helper routing. + pub(crate) fn set_callable_name_current_module_path(&mut self, path: Vec) { + self.callable_name_current_module_path = path; + } + + /// Configure the canonical callable registry for explicit cross-module call paths. + pub(crate) fn set_canonical_function_registry(&mut self, registry: FunctionRegistry) { + self.canonical_function_registry = Some(registry); + } + + pub(super) fn canonical_function_registry(&self) -> &FunctionRegistry { + self.canonical_function_registry + .as_ref() + .unwrap_or(self.function_registry) + } + + /// Configure the concrete callable-name helper modules available to this emitter. + pub(crate) fn set_callable_name_resolutions(&mut self, resolutions: HashMap) { + self.callable_name_resolutions = resolutions; + } + + /// Configure the callable-name signatures that are used anywhere in this generated crate. + pub(crate) fn set_callable_name_used_signature_keys(&mut self, keys: HashSet) { + self.callable_name_used_signature_keys = keys; + } + + /// Configure the local callable registry used by generated callable-name helpers. + pub(crate) fn set_callable_name_local_registry(&mut self, registry: FunctionRegistry) { + self.callable_name_local_registry = Some(registry); + } + + /// Add every concrete function-pointer signature from one lowered program to the cross-module resolver map. + pub(crate) fn add_callable_name_resolutions_for_program( + out: &mut HashMap, + module_path: Vec, + program: &IrProgram, + ) { + for (_, signature) in program.function_registry.iter() { + let params = signature + .params + .iter() + .map(|param| param.ty.clone()) + .collect::>(); + let ret = signature.return_type.clone(); + let Some(key) = Self::callable_name_signature_key(¶ms, &ret) else { + continue; + }; + let resolution = out.entry(key).or_insert_with(|| CallableNameResolution { + params, + ret, + module_paths: Vec::new(), + }); + if !resolution.module_paths.contains(&module_path) { + resolution.module_paths.push(module_path.clone()); + } + } + for resolution in out.values_mut() { + resolution.module_paths.sort(); + } + } + + /// Return the deterministic helper identifier for a concrete callable signature key. + pub(super) fn callable_name_helper_ident(key: &str) -> proc_macro2::Ident { + format_ident!( + "__incan_callable_name_{:016x}", + Self::stable_callable_name_hash(key.as_bytes()) + ) + } + + /// Return a stable signature key for callable-name helpers when the function-pointer type is concrete. + pub(super) fn callable_name_signature_key(params: &[IrType], ret: &IrType) -> Option { + if !params.iter().all(Self::callable_name_type_supported) || !Self::callable_name_type_supported(ret) { + return None; + } + let params = params.iter().map(IrType::rust_name).collect::>().join(", "); + Some(format!("fn({params}) -> {}", ret.rust_name())) + } + + fn callable_name_signature_key_from_signature(signature: &FunctionSignature) -> Option { + let params = signature + .params + .iter() + .map(|param| param.ty.clone()) + .collect::>(); + Self::callable_name_signature_key(¶ms, &signature.return_type) + } + + fn callable_name_type_supported(ty: &IrType) -> bool { + match ty { + IrType::Unknown | IrType::Generic(_) | IrType::ImplTrait(_) | IrType::SelfType => false, + IrType::List(inner) + | IrType::Set(inner) + | IrType::Option(inner) + | IrType::Ref(inner) + | IrType::RefMut(inner) => Self::callable_name_type_supported(inner), + IrType::Dict(key, value) | IrType::Result(key, value) => { + Self::callable_name_type_supported(key) && Self::callable_name_type_supported(value) + } + IrType::Tuple(items) => items.iter().all(Self::callable_name_type_supported), + IrType::NamedGeneric(_, args) => args.iter().all(Self::callable_name_type_supported), + IrType::Function { params, ret } => Self::callable_name_signature_key(params, ret).is_some(), + IrType::Unit + | IrType::Bool + | IrType::Int + | IrType::Float + | IrType::Decimal { .. } + | IrType::String + | IrType::StrRef + | IrType::StaticStr + | IrType::FrozenStr + | IrType::Bytes + | IrType::StaticBytes + | IrType::FrozenBytes + | IrType::Numeric(_) + | IrType::Struct(_) + | IrType::Enum(_) + | IrType::Trait(_) => true, + } + } + + fn stable_callable_name_hash(bytes: &[u8]) -> u64 { + let mut hash = 0xcbf29ce484222325u64; + for byte in bytes { + hash ^= u64::from(*byte); + hash = hash.wrapping_mul(0x100000001b3); + } + hash + } + + pub(super) fn local_callable_name_signature_keys(&self) -> HashSet { + self.callable_name_local_registry() + .iter() + .filter_map(|(_, signature)| Self::callable_name_signature_key_from_signature(signature)) + .collect() + } + + pub(super) fn callable_name_local_registry(&self) -> &FunctionRegistry { + self.callable_name_local_registry + .as_ref() + .unwrap_or(self.function_registry) + } + + /// Return whether two call-signature types describe the same emitted surface after transparent aliases expand. + pub(in crate::backend::ir::emit) fn call_signature_type_matches(&self, left: &IrType, right: &IrType) -> bool { + left == right || self.resolve_type_aliases_for_emit(left) == self.resolve_type_aliases_for_emit(right) + } + + /// Resolve transparent type aliases before emission decisions that need structural type information. + pub(in crate::backend::ir::emit) fn resolve_type_aliases_for_emit(&self, ty: &IrType) -> IrType { + let mut visiting = HashSet::new(); + self.resolve_type_aliases_for_emit_inner(ty, &mut visiting) + } + + /// Resolve nested transparent aliases while preserving cycles as their original alias names. + fn resolve_type_aliases_for_emit_inner(&self, ty: &IrType, visiting: &mut HashSet) -> IrType { + match ty { + IrType::Struct(name) | IrType::NamedGeneric(name, _) if self.type_aliases.contains_key(name) => { + if !visiting.insert(name.clone()) { + return ty.clone(); + } + let Some(target) = self.type_aliases.get(name) else { + visiting.remove(name); + return ty.clone(); + }; + let resolved = self.resolve_type_aliases_for_emit_inner(target, visiting); + visiting.remove(name); + resolved + } + IrType::List(inner) => IrType::List(Box::new(self.resolve_type_aliases_for_emit_inner(inner, visiting))), + IrType::Dict(key, value) => IrType::Dict( + Box::new(self.resolve_type_aliases_for_emit_inner(key, visiting)), + Box::new(self.resolve_type_aliases_for_emit_inner(value, visiting)), + ), + IrType::Set(inner) => IrType::Set(Box::new(self.resolve_type_aliases_for_emit_inner(inner, visiting))), + IrType::Tuple(items) => IrType::Tuple( + items + .iter() + .map(|item| self.resolve_type_aliases_for_emit_inner(item, visiting)) + .collect(), + ), + IrType::Option(inner) => { + IrType::Option(Box::new(self.resolve_type_aliases_for_emit_inner(inner, visiting))) + } + IrType::Result(ok, err) => IrType::Result( + Box::new(self.resolve_type_aliases_for_emit_inner(ok, visiting)), + Box::new(self.resolve_type_aliases_for_emit_inner(err, visiting)), + ), + IrType::NamedGeneric(name, args) => IrType::NamedGeneric( + name.clone(), + args.iter() + .map(|arg| self.resolve_type_aliases_for_emit_inner(arg, visiting)) + .collect(), + ), + IrType::Function { params, ret } => IrType::Function { + params: params + .iter() + .map(|param| self.resolve_type_aliases_for_emit_inner(param, visiting)) + .collect(), + ret: Box::new(self.resolve_type_aliases_for_emit_inner(ret, visiting)), + }, + IrType::Ref(inner) => IrType::Ref(Box::new(self.resolve_type_aliases_for_emit_inner(inner, visiting))), + IrType::RefMut(inner) => { + IrType::RefMut(Box::new(self.resolve_type_aliases_for_emit_inner(inner, visiting))) + } + _ => ty.clone(), } } pub(super) fn emit_module_static_init_call(&self) -> TokenStream { - if *self.module_has_local_statics.borrow() { + if *self.module_has_local_statics.borrow() || !self.imported_static_module_init_bindings.borrow().is_empty() { let init_fn = Self::rust_ident("__incan_init_module_statics"); quote! { #init_fn(); } } else { @@ -361,6 +642,48 @@ impl<'a> IrEmitter<'a> { } } + pub(super) fn set_imported_static_init_bindings(&self, bindings: HashSet) { + *self.imported_static_init_bindings.borrow_mut() = bindings; + } + + pub(super) fn set_imported_static_module_init_bindings(&self, bindings: Vec) { + *self.imported_static_module_init_bindings.borrow_mut() = bindings; + } + + pub(super) fn imported_static_init_ident(name: &str) -> proc_macro2::Ident { + let mut rendered = String::from("__incan_init_imported_static_"); + for ch in name.chars() { + if ch.is_ascii_alphanumeric() { + rendered.push(ch.to_ascii_lowercase()); + } else { + rendered.push('_'); + } + } + proc_macro2::Ident::new(&rendered, proc_macro2::Span::call_site()) + } + + pub(super) fn static_needs_imported_init_call(&self, name: &str) -> bool { + self.imported_static_init_bindings.borrow().contains(name) + } + + pub(super) fn static_needs_imported_init_import(&self, name: &str) -> bool { + self.static_needs_imported_init_call(name) + || self + .imported_static_module_init_bindings + .borrow() + .iter() + .any(|binding| binding == name) + } + + pub(super) fn emit_static_init_call_for_static(&self, name: &str) -> TokenStream { + if self.static_needs_imported_init_call(name) { + let init_fn = Self::imported_static_init_ident(name); + quote! { #init_fn(); } + } else { + self.emit_module_static_init_call() + } + } + /// Return the private helper method name used to call callable-object observers through a borrowed payload. pub(super) fn result_observer_borrowed_method_name() -> &'static str { "__incan_result_observer_borrow___call__" @@ -564,14 +887,12 @@ impl<'a> IrEmitter<'a> { /// True when a method should be emitted for a preserved public surface or an observed generated-use call. pub(super) fn should_emit_method(&self, target_type: &str, method_name: &str, visibility: &Visibility) -> bool { - let analysis = self.generated_use_analysis.borrow(); - analysis.public_types.contains(target_type) - || (!self.preserve_public_items - && !matches!(visibility, Visibility::Private) - && analysis.reachable_items.contains(target_type)) - || analysis - .used_methods - .contains(&(target_type.to_string(), method_name.to_string())) + self.generated_use_analysis.borrow().should_retain_method( + self.preserve_public_items, + target_type, + method_name, + visibility, + ) } /// True when the generated free constructor function for a struct should be retained. @@ -607,6 +928,47 @@ impl<'a> IrEmitter<'a> { self.ambiguous_type_names = ambiguous; } + /// Set value-to-module path mappings for dependency expressions that must be emitted outside their defining + /// module. + pub fn set_value_module_paths(&mut self, paths: HashMap>, ambiguous: HashSet) { + self.value_module_paths = paths; + self.ambiguous_value_names = ambiguous; + } + + pub(in crate::backend::ir::emit) fn emit_dependency_item_path( + &self, + module_path: &[String], + name: &str, + ) -> Option { + let mut segments = vec![quote! { crate }]; + for segment in module_path { + let ident = Self::rust_ident(segment); + segments.push(quote! { #ident }); + } + let ident = Self::rust_ident(name); + segments.push(quote! { #ident }); + + let mut iter = segments.into_iter(); + let first = iter.next()?; + Some(iter.fold(first, |acc, segment| quote! { #acc :: #segment })) + } + + pub(in crate::backend::ir::emit) fn emit_dependency_type_path(&self, name: &str) -> Option { + if name.contains("::") || self.ambiguous_type_names.contains(name) { + return None; + } + let module_path = self.type_module_paths.get(name)?; + self.emit_dependency_item_path(module_path, name) + } + + pub(in crate::backend::ir::emit) fn emit_dependency_value_path(&self, name: &str) -> Option { + if name.contains("::") || self.ambiguous_value_names.contains(name) { + return None; + } + let module_path = self.value_module_paths.get(name)?; + self.emit_dependency_item_path(module_path, name) + } + /// Set imported enum type names discovered during codegen setup. pub fn set_dependency_enum_types(&mut self, enum_type_names: HashSet) { self.dependency_enum_types = enum_type_names; @@ -676,13 +1038,20 @@ impl<'a> IrEmitter<'a> { } IrDeclKind::TypeAlias { name, - is_rusttype: true, + type_params, + ty, + is_rusttype, .. } => { if skip_ambiguous && self.ambiguous_type_names.contains(name) { continue; } - self.rusttype_alias_names.insert(name.clone()); + if type_params.is_empty() && !is_rusttype { + self.type_aliases.insert(name.clone(), ty.clone()); + } + if *is_rusttype { + self.rusttype_alias_names.insert(name.clone()); + } } IrDeclKind::Impl(i) => { for method in &i.methods { diff --git a/src/backend/ir/emit/program.rs b/src/backend/ir/emit/program.rs index 7e8684ff6..9ba889be0 100644 --- a/src/backend/ir/emit/program.rs +++ b/src/backend/ir/emit/program.rs @@ -20,7 +20,7 @@ use proc_macro2::TokenStream; use quote::{format_ident, quote}; -use std::collections::{HashMap, HashSet}; +use std::collections::{BTreeMap, HashMap, HashSet}; use incan_core::lang::surface::result_methods::ResultMethodId; use incan_core::lang::traits::{self as core_traits, TraitId}; @@ -36,7 +36,7 @@ use super::super::expr::{ use super::super::stmt::AssignTarget; use super::super::types::{IR_UNION_TYPE_NAME, IrType}; use super::super::{FunctionRegistry, FunctionSignature, IrDecl, IrProgram, IrStmt, IrStmtKind, TypedExpr}; -use super::{EmitError, GeneratedUseAnalysis, IrEmitter}; +use super::{CallableNameUseFacts, EmitError, GeneratedUseAnalysis, IrEmitter}; struct OrdinalValueEnumBridgeSpec { type_path: TokenStream, @@ -343,25 +343,23 @@ impl<'program> GeneratedUseAnalyzer<'program> { match magic_methods::from_str(method.name.as_str()) { Some(magic_methods::MagicMethodId::Eq | magic_methods::MagicMethodId::Str) => true, Some(magic_methods::MagicMethodId::ClassName | magic_methods::MagicMethodId::Fields) => { - self.method_is_needed(&impl_block.target_type, method) + self.analysis.should_retain_method( + self.preserve_public_items, + &impl_block.target_type, + &method.name, + &method.visibility, + ) } _ if impl_block.trait_name.is_some() => true, - _ => self.method_is_needed(&impl_block.target_type, method), + _ => self.analysis.should_retain_method( + self.preserve_public_items, + &impl_block.target_type, + &method.name, + &method.visibility, + ), } } - /// Mirror the emitter's method-retention predicate for generated-use analysis. - fn method_is_needed(&self, target_type: &str, method: &IrFunction) -> bool { - self.analysis.public_types.contains(target_type) - || (!self.preserve_public_items - && !matches!(method.visibility, Visibility::Private) - && self.analysis.reachable_items.contains(target_type)) - || self - .analysis - .used_methods - .contains(&(target_type.to_string(), method.name.clone())) - } - /// Scan a function signature, defaults, and body for generated Rust dependencies. fn scan_function(&mut self, func: &IrFunction) { let outer_variable_types = std::mem::take(&mut self.variable_types); @@ -573,6 +571,9 @@ impl<'program> GeneratedUseAnalyzer<'program> { self.scan_type(ty); } for arg in args { + for key in self.callable_name_function_arg_signature_keys(&arg.expr) { + self.analysis.callable_name_function_arg_signature_keys.insert(key); + } self.scan_expr(&arg.expr); } } @@ -622,6 +623,15 @@ impl<'program> GeneratedUseAnalyzer<'program> { } IrExprKind::Field { object, field } => { self.scan_expr(object); + if field == "__name__" + && let IrType::Function { params, ret } = &object.ty + && let Some(key) = IrEmitter::callable_name_signature_key(params, ret) + { + self.analysis.callable_name_signature_keys.insert(key); + } + if field == "__name__" && matches!(object.ty, IrType::Generic(_)) { + self.analysis.uses_generic_callable_name_trait = true; + } if let Some(type_name) = self.object_nominal_type_name(object) { let field = self .struct_field_aliases @@ -763,7 +773,7 @@ impl<'program> GeneratedUseAnalyzer<'program> { } IrExprKind::Format { parts } => { for part in parts { - if let super::super::expr::FormatPart::Expr(expr) = part { + if let super::super::expr::FormatPart::Expr { expr, .. } = part { self.scan_expr(expr); } } @@ -784,6 +794,52 @@ impl<'program> GeneratedUseAnalyzer<'program> { } } + fn callable_name_function_arg_signature_keys(&self, expr: &TypedExpr) -> Vec { + match &expr.kind { + IrExprKind::Var { name, .. } => { + let mut keys = HashSet::new(); + if let IrType::Function { params, ret } = &expr.ty + && let Some(key) = IrEmitter::callable_name_signature_key(params, ret) + { + keys.insert(key); + } + if let Some(signature) = self.function_registry.get(name) { + let params = signature + .params + .iter() + .map(|param| param.ty.clone()) + .collect::>(); + if let Some(key) = IrEmitter::callable_name_signature_key(¶ms, &signature.return_type) { + keys.insert(key); + } + } + let Some(IrDecl { + kind: IrDeclKind::Function(func), + .. + }) = self.declarations_by_name.get(name).copied() + else { + let mut keys = keys.into_iter().collect::>(); + keys.sort(); + return keys; + }; + if func.is_async || !func.type_params.is_empty() { + return Vec::new(); + } + let params = func.params.iter().map(|param| param.ty.clone()).collect::>(); + if let Some(key) = IrEmitter::callable_name_signature_key(¶ms, &func.return_type) { + keys.insert(key); + } + let mut keys = keys.into_iter().collect::>(); + keys.sort(); + keys + } + IrExprKind::InteropCoerce { expr, .. } + | IrExprKind::NumericResize { expr, .. } + | IrExprKind::Cast { expr, .. } => self.callable_name_function_arg_signature_keys(expr), + _ => Vec::new(), + } + } + /// Record non-Copy observer callbacks that need generated borrowed helper items. fn record_result_observer_callback( &mut self, @@ -831,29 +887,14 @@ impl<'program> GeneratedUseAnalyzer<'program> { IrExprKind::Var { name, .. } => Some(name.as_str()), _ => None, }; - let canonical_name = canonical_path.as_ref().and_then(|path| path.last()).map(String::as_str); - local_name - .and_then(|name| self.function_registry.get(name).cloned()) - .or_else(|| canonical_name.and_then(|name| self.function_registry.get(name).cloned())) - .or_else(|| callable_signature.cloned()) - .or_else(|| match &func.ty { - IrType::Function { params, ret } => Some(FunctionSignature { - params: params - .iter() - .enumerate() - .map(|(idx, ty)| super::super::decl::FunctionParam { - name: format!("__incan_arg_{idx}"), - ty: ty.clone(), - mutability: super::super::types::Mutability::Immutable, - is_self: false, - kind: crate::frontend::ast::ParamKind::Normal, - default: None, - }) - .collect(), - return_type: ret.as_ref().clone(), - }), - _ => None, - }) + FunctionRegistry::effective_call_signature( + self.function_registry, + self.function_registry, + local_name, + canonical_path.as_deref(), + callable_signature, + Some(&func.ty), + ) } /// Record named function arguments that need private adapters for borrowed function-pointer parameters. @@ -916,16 +957,15 @@ impl<'program> GeneratedUseAnalyzer<'program> { method: &str, dispatch: Option<&IrMethodDispatch>, ) { - if let Some(IrMethodDispatch::RustExtensionTraitImport { binding }) = dispatch { - if self.rust_extension_trait_imports.contains_key(binding) { - self.analysis.used_extension_trait_imports.insert(binding.clone()); + let Some(IrMethodDispatch::RustExtensionTraitImport { binding }) = dispatch else { + if self.receiver_can_use_rust_extension_trait(receiver) { + self.mark_unambiguous_rust_extension_trait_import(method); } return; + }; + if self.rust_extension_trait_imports.contains_key(binding) { + self.analysis.used_extension_trait_imports.insert(binding.clone()); } - if !self.receiver_can_use_rust_extension_trait(receiver) { - return; - } - self.mark_unambiguous_rust_extension_trait_import(method); } /// Mark a trait import for metadata-free fallback only when the method has one possible imported trait. @@ -1091,6 +1131,44 @@ impl<'program> GeneratedUseAnalyzer<'program> { } impl<'a> IrEmitter<'a> { + fn collect_imported_static_init_bindings(&self, declarations: &[&IrDecl]) -> (HashSet, Vec) { + let mut access_bindings = HashSet::new(); + let mut module_init_bindings = HashSet::new(); + for decl in declarations { + let IrDeclKind::Import { + visibility, + origin, + qualifier, + path, + items, + .. + } = &decl.kind + else { + continue; + }; + if matches!(origin, IrImportOrigin::PubLibrary { .. }) || matches!(qualifier, IrImportQualifier::None) { + continue; + } + let is_incan_source_stdlib = Self::is_incan_source_stdlib_import(origin, qualifier, path); + let is_public_reexport = !matches!(visibility, Visibility::Private); + for item in items { + if !item.is_static { + continue; + } + let binding = item.alias.as_ref().unwrap_or(&item.name); + if self.should_emit_import_binding(binding) { + access_bindings.insert(binding.clone()); + } + if is_public_reexport && !(is_incan_source_stdlib && binding.starts_with('_')) { + module_init_bindings.insert(binding.clone()); + } + } + } + let mut module_init_bindings: Vec<_> = module_init_bindings.into_iter().collect(); + module_init_bindings.sort(); + (access_bindings, module_init_bindings) + } + /// Return whether the current emitted module defines one registry-backed temporary capability trait contract. fn emitted_declarations_define_capability_trait( program: &IrProgram, @@ -1792,7 +1870,7 @@ impl<'a> IrEmitter<'a> { } IrExprKind::Format { parts } => { for part in parts { - if let super::super::expr::FormatPart::Expr(expr) = part { + if let super::super::expr::FormatPart::Expr { expr, .. } = part { Self::collect_union_types_from_expr(expr, out); } } @@ -2018,13 +2096,13 @@ impl<'a> IrEmitter<'a> { fn emit_generated_union_member_type(&self, ty: &IrType) -> TokenStream { match ty { IrType::Struct(name) | IrType::Enum(name) | IrType::Trait(name) => self - .emit_dependency_nominal_type_path(name) + .emit_dependency_type_path(name) .unwrap_or_else(|| self.emit_type(ty)), IrType::NamedGeneric(name, args) if name == super::super::types::IR_UNION_TYPE_NAME => { self.emit_union_type_path(ty) } IrType::NamedGeneric(name, args) => { - let base = self.emit_dependency_nominal_type_path(name).unwrap_or_else(|| { + let base = self.emit_dependency_type_path(name).unwrap_or_else(|| { let ident = Self::rust_ident(name); quote! { #ident } }); @@ -2099,25 +2177,6 @@ impl<'a> IrEmitter<'a> { } } - /// Emit a crate-qualified path for an unambiguous nominal type declared in a dependency module. - fn emit_dependency_nominal_type_path(&self, name: &str) -> Option { - if name.contains("::") || self.ambiguous_type_names.contains(name) { - return None; - } - let module_path = self.type_module_paths.get(name)?; - let mut segments = vec![quote! { crate }]; - for segment in module_path { - let ident = Self::rust_ident(segment); - segments.push(quote! { #ident }); - } - let name_ident = Self::rust_ident(name); - segments.push(quote! { #name_ident }); - - let mut iter = segments.into_iter(); - let first = iter.next()?; - Some(iter.fold(first, |acc, segment| quote! { #acc :: #segment })) - } - /// Emit a complete IR program to formatted Rust code. #[tracing::instrument(skip_all, fields(decl_count = program.declarations.len()))] pub fn emit_program(&mut self, program: &IrProgram) -> Result { @@ -2163,6 +2222,18 @@ impl<'a> IrEmitter<'a> { .insert((e.name.clone(), alias.name.clone()), alias.target.clone()); } } + if let IrDeclKind::TypeAlias { + name, + type_params, + ty, + is_rusttype, + .. + } = &decl.kind + && type_params.is_empty() + && !is_rusttype + { + self.type_aliases.insert(name.clone(), ty.clone()); + } if let IrDeclKind::TypeAlias { name, is_rusttype: true, @@ -2214,6 +2285,230 @@ impl<'a> IrEmitter<'a> { Ok(format!("{}{}", header, with_marker)) } + pub(crate) fn callable_name_use_facts_for_program( + program: &IrProgram, + externally_reachable_items: &HashSet, + preserve_public_items: bool, + external_error_trait_types: &HashSet, + ) -> CallableNameUseFacts { + let analysis = GeneratedUseAnalyzer::analyze( + program, + externally_reachable_items, + preserve_public_items, + external_error_trait_types, + ); + CallableNameUseFacts { + signature_keys: analysis.callable_name_signature_keys, + function_arg_signature_keys: analysis.callable_name_function_arg_signature_keys, + generic_trait_used: analysis.uses_generic_callable_name_trait, + } + } + + fn callable_name_signature_for_key(&self, key: &str) -> Option<(Vec, IrType)> { + self.callable_name_local_registry() + .iter() + .find_map(|(_, signature)| { + let params = signature + .params + .iter() + .map(|param| param.ty.clone()) + .collect::>(); + (Self::callable_name_signature_key(¶ms, &signature.return_type).as_deref() == Some(key)) + .then(|| (params, signature.return_type.clone())) + }) + .or_else(|| { + self.callable_name_resolutions + .get(key) + .map(|resolution| (resolution.params.clone(), resolution.ret.clone())) + }) + } + + fn callable_name_helper_keys( + &self, + local_callable_name_signature_keys: &HashSet, + include_generic_callable_signatures: bool, + ) -> Vec { + let mut keys = local_callable_name_signature_keys.clone(); + if include_generic_callable_signatures { + keys.extend(self.callable_name_used_signature_keys.iter().filter_map(|key| { + self.callable_name_signature_for_key(key) + .is_some() + .then_some(key.clone()) + })); + } + for (key, resolution) in &self.callable_name_resolutions { + if self.callable_name_used_signature_keys.contains(key) + && resolution + .module_paths + .contains(&self.callable_name_current_module_path) + { + keys.insert(key.clone()); + } + } + let mut keys = keys.into_iter().collect::>(); + keys.sort(); + keys + } + + fn callable_name_resolution_expr_with_fallback( + &self, + key: &str, + callable_tokens: TokenStream, + fallback: TokenStream, + ) -> TokenStream { + let helper = Self::callable_name_helper_ident(key); + let mut helper_calls = Vec::new(); + helper_calls.push(quote! { #helper(#callable_tokens) }); + if let Some(resolution) = self.callable_name_resolutions.get(key) { + for module_path in &resolution.module_paths { + if module_path == &self.callable_name_current_module_path { + continue; + } + if module_path.is_empty() && !self.callable_name_current_module_path.is_empty() { + continue; + } + let helper_path = self.emit_callable_name_helper_path(module_path, key); + helper_calls.push(quote! { #helper_path(#callable_tokens) }); + } + } + let mut resolved = fallback; + for helper_call in helper_calls.into_iter().rev() { + resolved = quote! { + if let Some(__incan_name) = #helper_call { + __incan_name.to_string() + } else { + #resolved + } + }; + } + resolved + } + + fn emit_generic_callable_name_trait(&self, keys: &[String]) -> Option { + if keys.is_empty() { + return None; + } + let trait_ident = Self::rust_ident("__IncanCallableName"); + let mut grouped_keys: BTreeMap> = BTreeMap::new(); + for key in keys { + let Some((params, ret)) = self.callable_name_signature_for_key(key) else { + continue; + }; + let resolved_params = params + .iter() + .map(|param| self.resolve_type_aliases_for_emit(param)) + .collect::>(); + let resolved_ret = self.resolve_type_aliases_for_emit(&ret); + let Some(resolved_key) = Self::callable_name_signature_key(&resolved_params, &resolved_ret) else { + continue; + }; + grouped_keys.entry(resolved_key).or_default().push(key.clone()); + } + + let impls = grouped_keys + .values_mut() + .filter_map(|keys| { + keys.sort(); + let primary_key = keys.first()?; + let (params, ret) = self.callable_name_signature_for_key(primary_key)?; + let fn_ty = self.emit_callable_fn_type(¶ms, &ret); + let fallback = proc_macro2::Literal::string(""); + let mut resolved = quote! { #fallback.to_string() }; + for key in keys.iter().rev() { + resolved = + self.callable_name_resolution_expr_with_fallback(key, quote! { __incan_callable }, resolved); + } + Some(quote! { + impl #trait_ident for #fn_ty { + fn __incan_callable_name(&self) -> String { + let __incan_callable: #fn_ty = *self; + #resolved + } + } + }) + }) + .collect::>(); + if impls.is_empty() { + return None; + } + Some(quote! { + pub trait #trait_ident { + fn __incan_callable_name(&self) -> String; + } + + #(#impls)* + }) + } + + fn emit_callable_name_helpers( + &self, + emitted_callable_names: &HashSet, + keys: &[String], + ) -> Vec { + keys.iter() + .filter_map(|key| { + let (params, ret) = self.callable_name_signature_for_key(key)?; + let helper = Self::callable_name_helper_ident(key); + let fn_ty = self.emit_callable_fn_type(¶ms, &ret); + let mut candidates = self + .callable_name_local_registry() + .iter() + .filter(|(name, signature)| { + emitted_callable_names.contains(*name) + && signature.params.len() == params.len() + && signature.params.iter().map(|param| ¶m.ty).eq(params.iter()) + && signature.return_type == ret + }) + .map(|(name, _)| { + let source_name = name.strip_prefix("__incan_original_").unwrap_or(name); + (name.clone(), source_name.to_string()) + }) + .collect::>(); + candidates.sort_by(|left, right| left.0.cmp(&right.0)); + let has_candidates = !candidates.is_empty(); + + let mut body = quote! { None }; + for (candidate, source_name) in candidates.into_iter().rev() { + let candidate_ident = Self::rust_ident(&candidate); + let source_literal = proc_macro2::Literal::string(&source_name); + body = quote! { + if std::ptr::fn_addr_eq(callable, #candidate_ident as #fn_ty) { + Some(#source_literal) + } else { + #body + } + }; + } + let callable_param = if has_candidates { + Self::rust_ident("callable") + } else { + Self::rust_ident("_callable") + }; + + let visibility = if self.callable_name_resolutions.get(key).is_some_and(|resolution| { + self.callable_name_used_signature_keys.contains(key) + && resolution + .module_paths + .contains(&self.callable_name_current_module_path) + }) { + quote! { pub(crate) } + } else { + quote! {} + }; + let private_interfaces_allow = (!visibility.is_empty()).then(|| { + quote! { #[allow(private_interfaces)] } + }); + + Some(quote! { + #private_interfaces_allow + #visibility fn #helper(#callable_param: #fn_ty) -> Option<&'static str> { + #body + } + }) + }) + .collect() + } + /// Emit a program to TokenStream (without formatting). pub fn emit_program_tokens(&self, program: &IrProgram) -> Result { let mut items = Vec::new(); @@ -2226,9 +2521,13 @@ impl<'a> IrEmitter<'a> { let uses_stdlib_error_trait = analysis.uses_stdlib_error_trait; let result_observer_callable_types = analysis.result_observer_callable_types.clone(); let borrowed_function_adapters = analysis.borrowed_function_adapters.clone(); + let local_callable_name_signature_keys = analysis.callable_name_signature_keys.clone(); + let uses_generic_callable_name_trait = analysis.uses_generic_callable_name_trait; self.set_result_observer_callable_types(result_observer_callable_types); self.set_borrowed_function_adapters(borrowed_function_adapters); self.set_generated_use_analysis(analysis); + let callable_name_helper_keys = + self.callable_name_helper_keys(&local_callable_name_signature_keys, uses_generic_callable_name_trait); let emitted_declarations: Vec<&IrDecl> = program .declarations @@ -2243,6 +2542,10 @@ impl<'a> IrEmitter<'a> { }) .collect(); *self.module_has_local_statics.borrow_mut() = !static_names.is_empty(); + let (imported_static_init_bindings, imported_static_module_init_bindings) = + self.collect_imported_static_init_bindings(&emitted_declarations); + self.set_imported_static_init_bindings(imported_static_init_bindings); + self.set_imported_static_module_init_bindings(imported_static_module_init_bindings); if self.emit_strict_generated_lint_denies { items.push(quote! { @@ -2261,16 +2564,20 @@ impl<'a> IrEmitter<'a> { matches!( &decl.kind, IrDeclKind::Impl(impl_block) - if impl_block.trait_name.as_deref() == Some("json.Serialize") - || impl_block.trait_name.as_deref() == Some("std.serde.json.Serialize") + if impl_block.trait_name + .as_deref() + .and_then(incan_core::lang::stdlib::stdlib_json_trait_scope_import_id) + == Some(incan_core::lang::stdlib::StdlibJsonTraitId::Serialize) ) }); let needs_json_deserialize_trait_scope = emitted_declarations.iter().any(|decl| { matches!( &decl.kind, IrDeclKind::Impl(impl_block) - if impl_block.trait_name.as_deref() == Some("json.Deserialize") - || impl_block.trait_name.as_deref() == Some("std.serde.json.Deserialize") + if impl_block.trait_name + .as_deref() + .and_then(incan_core::lang::stdlib::stdlib_json_trait_scope_import_id) + == Some(incan_core::lang::stdlib::StdlibJsonTraitId::Deserialize) ) }); match (needs_json_serialize_trait_scope, needs_json_deserialize_trait_scope) { @@ -2325,7 +2632,16 @@ impl<'a> IrEmitter<'a> { } // RFC 052: force declaration-order static initialization once per module before any static access helper call. - if !static_names.is_empty() { + let imported_static_init_calls: Vec = self + .imported_static_module_init_bindings + .borrow() + .iter() + .map(|name| { + let ident = Self::imported_static_init_ident(name); + quote! { #ident(); } + }) + .collect(); + if !static_names.is_empty() || !imported_static_init_calls.is_empty() { let force_calls: Vec = static_names .iter() .map(|name| { @@ -2335,7 +2651,7 @@ impl<'a> IrEmitter<'a> { .collect(); items.push(quote! { #[inline(always)] - fn __incan_init_module_statics() { + pub(crate) fn __incan_init_module_statics() { static __INCAN_STATIC_INIT_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); if __INCAN_STATIC_INIT_RUNNING.load(std::sync::atomic::Ordering::Acquire) { @@ -2351,12 +2667,28 @@ impl<'a> IrEmitter<'a> { } __INCAN_STATIC_INIT_RUNNING.store(true, std::sync::atomic::Ordering::Release); let _guard = __IncanStaticInitGuard(&__INCAN_STATIC_INIT_RUNNING); + #(#imported_static_init_calls)* #(#force_calls)* }); } }); } + let emitted_callable_names: HashSet = emitted_declarations + .iter() + .filter_map(|decl| match &decl.kind { + IrDeclKind::Function(func) => Some(func.name.clone()), + IrDeclKind::SymbolAlias { name, .. } => Some(name.clone()), + _ => None, + }) + .collect(); + items.extend(self.emit_callable_name_helpers(&emitted_callable_names, &callable_name_helper_keys)); + if uses_generic_callable_name_trait + && let Some(trait_item) = self.emit_generic_callable_name_trait(&callable_name_helper_keys) + { + items.push(trait_item); + } + // Emit all declarations. let defines_ordinal_key_trait = Self::emitted_declarations_define_capability_trait( program, diff --git a/src/backend/ir/emit/statements.rs b/src/backend/ir/emit/statements.rs index 23e08fff4..c0bf6e20f 100644 --- a/src/backend/ir/emit/statements.rs +++ b/src/backend/ir/emit/statements.rs @@ -95,6 +95,15 @@ fn for_body_needs_mut_iteration(pattern: &Pattern, body: &[IrStmt]) -> bool { body.iter().any(|s| stmt_mutates_var(s, loop_var)) } +/// Return the element target type for assignment into a list index. +fn list_index_assignment_element_type(object_ty: &IrType) -> Option<&IrType> { + match object_ty { + IrType::Ref(inner) | IrType::RefMut(inner) => list_index_assignment_element_type(inner), + IrType::List(elem_ty) => Some(elem_ty.as_ref()), + _ => None, + } +} + /// Return the local `StaticBinding` name at the root of a storage-rooted expression. /// /// This is used by statement-slice analysis to detect aliases like `live` in @@ -764,7 +773,7 @@ fn expr_uses_binding_name(expr: &super::super::expr::IrExpr, binding_name: &str) .is_some_and(|expr| expr_uses_binding_name(expr, binding_name)) } IrExprKind::Format { parts } => parts.iter().any(|part| match part { - super::super::expr::FormatPart::Expr(expr) => expr_uses_binding_name(expr, binding_name), + super::super::expr::FormatPart::Expr { expr, .. } => expr_uses_binding_name(expr, binding_name), super::super::expr::FormatPart::Literal(_) => false, }), IrExprKind::Unit @@ -981,11 +990,11 @@ impl<'a> IrEmitter<'a> { let rhs_ident = format_ident!("{}", rhs_name); let rewritten_target = match target { AssignTarget::Field { object, field } => AssignTarget::Field { - object: Box::new(Self::rewrite_storage_root_expr(object, local_name)), + object: Box::new(Self::rewrite_storage_root_expr_for_mut(object, local_name)), field: field.clone(), }, AssignTarget::Index { object, index } => AssignTarget::Index { - object: Box::new(Self::rewrite_storage_root_expr(object, local_name)), + object: Box::new(Self::rewrite_storage_root_expr_for_mut(object, local_name)), index: index.clone(), }, _ => { @@ -1084,8 +1093,10 @@ impl<'a> IrEmitter<'a> { IrStmtKind::Assign { target, value } => { if let AssignTarget::Static(name) = target { let n = Self::rust_static_ident(name); + let init_call = self.emit_static_init_call_for_static(name); let v = self.emit_assignment_value(value, None)?; return Ok(quote! { + #init_call let __incan_static_rhs = #v; #n.with_mut(|__incan_static_value| { *__incan_static_value = __incan_static_rhs.into(); @@ -1133,6 +1144,18 @@ impl<'a> IrEmitter<'a> { .apply(v); return Ok(quote! { #o.insert(#k, #v); }); } + if let AssignTarget::Index { object, .. } = target + && let Some(value_target_ty) = list_index_assignment_element_type(&object.ty) + { + let t = self.emit_assign_target(target)?; + let v = self.emit_expr_for_use( + value, + ValueUseSite::Assignment { + target_ty: Some(value_target_ty), + }, + )?; + return Ok(quote! { #t = #v; }); + } let t = self.emit_assign_target(target)?; let v = self.emit_assignment_value(value, None)?; Ok(quote! { #t = #v; }) diff --git a/src/backend/ir/emit/types.rs b/src/backend/ir/emit/types.rs index 66876efad..6604a6401 100644 --- a/src/backend/ir/emit/types.rs +++ b/src/backend/ir/emit/types.rs @@ -122,6 +122,11 @@ impl<'a> IrEmitter<'a> { if name == surface_types::as_str(SurfaceTypeId::ValidationError) { return quote! { incan_stdlib::validation::ValidationError }; } + if *self.qualify_internal_canonical_paths.borrow() + && let Some(path) = self.emit_dependency_type_path(name) + { + return path; + } Self::emit_path_ident(name) } IrType::NamedGeneric(name, _) if name == super::super::types::IR_UNION_TYPE_NAME => { @@ -135,11 +140,15 @@ impl<'a> IrEmitter<'a> { Some(CollectionTypeId::Generator) => Some(quote! { incan_stdlib::iter::Generator }), _ => None, }; - let n = Self::emit_path_ident(name); let ts: Vec<_> = args.iter().map(|t| self.emit_type(t)).collect(); if let Some(n) = frozen_name { quote! { #n < #(#ts),* > } + } else if *self.qualify_internal_canonical_paths.borrow() + && let Some(n) = self.emit_dependency_type_path(name) + { + quote! { #n < #(#ts),* > } } else { + let n = Self::emit_path_ident(name); quote! { #n < #(#ts),* > } } } @@ -171,6 +180,14 @@ impl<'a> IrEmitter<'a> { } } + pub(in crate::backend::ir::emit) fn emit_callable_fn_type(&self, params: &[IrType], ret: &IrType) -> TokenStream { + let previous = self.qualify_internal_canonical_paths.replace(true); + let param_tokens = params.iter().map(|param| self.emit_type(param)).collect::>(); + let ret_tokens = self.emit_type(ret); + self.qualify_internal_canonical_paths.replace(previous); + quote! { fn(#(#param_tokens),*) -> #ret_tokens } + } + // ======================================================================== // RFC 023: Type parameter emission with trait bounds // ======================================================================== diff --git a/src/backend/ir/expr.rs b/src/backend/ir/expr.rs index 3a53d0fb1..48d172ad1 100644 --- a/src/backend/ir/expr.rs +++ b/src/backend/ir/expr.rs @@ -17,7 +17,9 @@ use super::decl::IrInteropAdapterKind; use super::{FunctionSignature, IrSpan, IrType, Ownership}; use incan_core::interop::CoercionPolicy; use incan_core::lang::builtins::{self as core_builtins, BuiltinFnId}; -use incan_core::lang::surface::{dict_methods, list_methods, result_methods, set_methods, string_methods}; +use incan_core::lang::surface::{ + dict_methods, iterator_methods, list_methods, result_methods, set_methods, string_methods, +}; use incan_core::lang::traits::{self as core_traits, TraitId}; use incan_core::lang::types::collections::{self as collection_types, CollectionTypeId}; @@ -417,7 +419,38 @@ pub enum FormatPart { /// Literal text Literal(String), /// Expression to interpolate - Expr(IrExpr), + Expr { expr: IrExpr, style: FormatStyle }, +} + +/// Formatting style requested by one f-string interpolation. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum FormatStyle { + /// User-facing display formatting (`{value}`). + #[default] + Display, + /// Structured debug formatting (`{value:?}`). + Debug, +} + +impl FormatStyle { + /// Return whether this interpolation should emit Rust debug formatting for the resolved backend type. + pub fn emits_rust_debug(self, ty: &IrType) -> bool { + matches!(self, Self::Debug) || matches!(self, Self::Display) && display_style_uses_structured_debug(ty) + } +} + +/// Return whether default Incan f-string display should use structured formatting for a backend representation that +/// does not expose Rust `Display` directly. +pub fn display_style_uses_structured_debug(ty: &IrType) -> bool { + matches!( + ty, + IrType::List(_) + | IrType::Dict(_, _) + | IrType::Set(_) + | IrType::Tuple(_) + | IrType::Option(_) + | IrType::Result(_, _) + ) } /// How a variable is accessed @@ -757,7 +790,7 @@ impl MethodKind { iterator_method_kind(name).map(Self::Iterator) } - /// Try to resolve an RFC 070 result-combinator method name without considering a receiver type. + /// Try to resolve a Result method name without considering a receiver type. pub fn for_result_method_name(name: &str) -> Option { result_methods::from_str(name).map(Self::Result) } @@ -798,7 +831,7 @@ impl MethodKind { })) } IrType::List(_) => { - if name == "iter" { + if iterator_methods::from_str(name) == Some(iterator_methods::IteratorMethodId::Iter) { return Some(Self::Iterator(IteratorMethodKind::Iter)); } let id = list_methods::from_str(name)?; @@ -828,7 +861,7 @@ impl MethodKind { })) } IrType::Set(_) => { - if name == "iter" { + if iterator_methods::from_str(name) == Some(iterator_methods::IteratorMethodId::Iter) { return Some(Self::Iterator(IteratorMethodKind::Iter)); } if set_methods::from_str(name).is_some() { @@ -840,7 +873,7 @@ impl MethodKind { if matches!( collection_types::from_str(type_name), Some(CollectionTypeId::FrozenList | CollectionTypeId::FrozenSet) - ) && name == "iter" => + ) && iterator_methods::from_str(name) == Some(iterator_methods::IteratorMethodId::Iter) => { Some(Self::Iterator(IteratorMethodKind::Iter)) } @@ -864,28 +897,30 @@ fn is_iterator_protocol_type_name(name: &str) -> bool { /// Classify an RFC 088 iterator method name into the structured backend method family. fn iterator_method_kind(name: &str) -> Option { - Some(match name { - "map" => IteratorMethodKind::Map, - "filter" => IteratorMethodKind::Filter, - "enumerate" => IteratorMethodKind::Enumerate, - "zip" => IteratorMethodKind::Zip, - "take" => IteratorMethodKind::Take, - "skip" => IteratorMethodKind::Skip, - "take_while" => IteratorMethodKind::TakeWhile, - "skip_while" => IteratorMethodKind::SkipWhile, - "chain" => IteratorMethodKind::Chain, - "flat_map" => IteratorMethodKind::FlatMap, - "batch" => IteratorMethodKind::Batch, - "collect" => IteratorMethodKind::Collect, - "count" => IteratorMethodKind::Count, - "reduce" => IteratorMethodKind::Reduce, - "fold" => IteratorMethodKind::Fold, - "any" => IteratorMethodKind::Any, - "all" => IteratorMethodKind::All, - "find" => IteratorMethodKind::Find, - "for_each" => IteratorMethodKind::ForEach, - "sum" => IteratorMethodKind::Sum, - _ => return None, + let id = iterator_methods::from_str(name)?; + use iterator_methods::IteratorMethodId as M; + Some(match id { + M::Iter => IteratorMethodKind::Iter, + M::Map => IteratorMethodKind::Map, + M::Filter => IteratorMethodKind::Filter, + M::Enumerate => IteratorMethodKind::Enumerate, + M::Zip => IteratorMethodKind::Zip, + M::Take => IteratorMethodKind::Take, + M::Skip => IteratorMethodKind::Skip, + M::TakeWhile => IteratorMethodKind::TakeWhile, + M::SkipWhile => IteratorMethodKind::SkipWhile, + M::Chain => IteratorMethodKind::Chain, + M::FlatMap => IteratorMethodKind::FlatMap, + M::Batch => IteratorMethodKind::Batch, + M::Collect => IteratorMethodKind::Collect, + M::Count => IteratorMethodKind::Count, + M::Reduce => IteratorMethodKind::Reduce, + M::Fold => IteratorMethodKind::Fold, + M::Any => IteratorMethodKind::Any, + M::All => IteratorMethodKind::All, + M::Find => IteratorMethodKind::Find, + M::ForEach => IteratorMethodKind::ForEach, + M::Sum => IteratorMethodKind::Sum, }) } @@ -949,7 +984,7 @@ mod tests { } #[test] - fn result_method_kind_for_receiver_classifies_rfc070_surface() { + fn result_method_kind_for_receiver_classifies_result_surface() { let result_ty = IrType::Result(Box::new(IrType::Int), Box::new(IrType::String)); for (name, expected) in [ ("map", result_methods::ResultMethodId::Map), @@ -958,6 +993,8 @@ mod tests { ("or_else", result_methods::ResultMethodId::OrElse), ("inspect", result_methods::ResultMethodId::Inspect), ("inspect_err", result_methods::ResultMethodId::InspectErr), + ("unwrap", result_methods::ResultMethodId::Unwrap), + ("unwrap_or", result_methods::ResultMethodId::UnwrapOr), ] { assert_eq!( MethodKind::for_receiver(&result_ty, name), @@ -965,6 +1002,6 @@ mod tests { "expected Result method classification for `{name}`" ); } - assert_eq!(MethodKind::for_receiver(&result_ty, "unwrap"), None); + assert_eq!(MethodKind::for_receiver(&result_ty, "missing"), None); } } diff --git a/src/backend/ir/lower/decl/functions.rs b/src/backend/ir/lower/decl/functions.rs index 1c6a6f829..716f9362f 100644 --- a/src/backend/ir/lower/decl/functions.rs +++ b/src/backend/ir/lower/decl/functions.rs @@ -2,6 +2,8 @@ use super::super::super::Mutability; use super::super::super::decl::{FunctionParam, IrFunction, IrTraitBound, IrTraitBoundOrigin}; +use super::super::super::expr::{IrDictEntry, IrExprKind, IrGeneratorClause, IrListEntry}; +use super::super::super::stmt::{AssignTarget, IrStmt, IrStmtKind}; use super::super::super::types::IrType; use super::super::AstLowering; use super::super::errors::LoweringError; @@ -31,6 +33,269 @@ fn body_contains_yield(body: &[ast::Spanned]) -> bool { }) } +fn collect_generic_callable_name_type_params_from_expr(expr: &super::super::super::IrExpr, out: &mut Vec) { + match &expr.kind { + IrExprKind::Field { object, field } => { + if field == "__name__" + && let IrType::Generic(name) = &object.ty + && !out.contains(name) + { + out.push(name.clone()); + } + collect_generic_callable_name_type_params_from_expr(object, out); + } + IrExprKind::BinOp { left, right, .. } => { + collect_generic_callable_name_type_params_from_expr(left, out); + collect_generic_callable_name_type_params_from_expr(right, out); + } + IrExprKind::UnaryOp { operand, .. } + | IrExprKind::Await(operand) + | IrExprKind::Try(operand) + | IrExprKind::NumericResize { expr: operand, .. } + | IrExprKind::Cast { expr: operand, .. } + | IrExprKind::InteropCoerce { expr: operand, .. } => { + collect_generic_callable_name_type_params_from_expr(operand, out); + } + IrExprKind::Call { func, args, .. } => { + collect_generic_callable_name_type_params_from_expr(func, out); + for arg in args { + collect_generic_callable_name_type_params_from_expr(&arg.expr, out); + } + } + IrExprKind::BuiltinCall { args, .. } => { + for arg in args { + collect_generic_callable_name_type_params_from_expr(arg, out); + } + } + IrExprKind::KnownMethodCall { args, .. } => { + for arg in args { + collect_generic_callable_name_type_params_from_expr(&arg.expr, out); + } + } + IrExprKind::MethodCall { receiver, args, .. } => { + collect_generic_callable_name_type_params_from_expr(receiver, out); + for arg in args { + collect_generic_callable_name_type_params_from_expr(&arg.expr, out); + } + } + IrExprKind::Index { object, index } => { + collect_generic_callable_name_type_params_from_expr(object, out); + collect_generic_callable_name_type_params_from_expr(index, out); + } + IrExprKind::Slice { + target, + start, + end, + step, + } => { + collect_generic_callable_name_type_params_from_expr(target, out); + for expr in [start, end, step].into_iter().flatten() { + collect_generic_callable_name_type_params_from_expr(expr, out); + } + } + IrExprKind::ListComp { + element, + iterable, + filter, + .. + } => { + collect_generic_callable_name_type_params_from_expr(element, out); + collect_generic_callable_name_type_params_from_expr(iterable, out); + if let Some(filter) = filter { + collect_generic_callable_name_type_params_from_expr(filter, out); + } + } + IrExprKind::DictComp { + key, + value, + iterable, + filter, + .. + } => { + collect_generic_callable_name_type_params_from_expr(key, out); + collect_generic_callable_name_type_params_from_expr(value, out); + collect_generic_callable_name_type_params_from_expr(iterable, out); + if let Some(filter) = filter { + collect_generic_callable_name_type_params_from_expr(filter, out); + } + } + IrExprKind::Generator { element, clauses } => { + collect_generic_callable_name_type_params_from_expr(element, out); + for clause in clauses { + match clause { + IrGeneratorClause::For { iterable, .. } => { + collect_generic_callable_name_type_params_from_expr(iterable, out); + } + IrGeneratorClause::If(condition) => { + collect_generic_callable_name_type_params_from_expr(condition, out); + } + } + } + } + IrExprKind::List(items) => { + for item in items { + match item { + IrListEntry::Element(value) | IrListEntry::Spread(value) => { + collect_generic_callable_name_type_params_from_expr(value, out); + } + } + } + } + IrExprKind::Dict(items) => { + for item in items { + match item { + IrDictEntry::Pair(key, value) => { + collect_generic_callable_name_type_params_from_expr(key, out); + collect_generic_callable_name_type_params_from_expr(value, out); + } + IrDictEntry::Spread(value) => { + collect_generic_callable_name_type_params_from_expr(value, out); + } + } + } + } + IrExprKind::Set(items) | IrExprKind::Tuple(items) => { + for item in items { + collect_generic_callable_name_type_params_from_expr(item, out); + } + } + IrExprKind::Struct { fields, .. } => { + for (_, value) in fields { + collect_generic_callable_name_type_params_from_expr(value, out); + } + } + IrExprKind::If { + condition, + then_branch, + else_branch, + } => { + collect_generic_callable_name_type_params_from_expr(condition, out); + collect_generic_callable_name_type_params_from_expr(then_branch, out); + if let Some(else_branch) = else_branch { + collect_generic_callable_name_type_params_from_expr(else_branch, out); + } + } + IrExprKind::Match { scrutinee, arms } => { + collect_generic_callable_name_type_params_from_expr(scrutinee, out); + for arm in arms { + if let Some(guard) = &arm.guard { + collect_generic_callable_name_type_params_from_expr(guard, out); + } + collect_generic_callable_name_type_params_from_expr(&arm.body, out); + } + } + IrExprKind::Closure { body, .. } => { + collect_generic_callable_name_type_params_from_expr(body, out); + } + IrExprKind::Block { stmts, value } => { + collect_generic_callable_name_type_params_from_stmts(stmts, out); + if let Some(value) = value { + collect_generic_callable_name_type_params_from_expr(value, out); + } + } + IrExprKind::Loop { body } => collect_generic_callable_name_type_params_from_stmts(body, out), + IrExprKind::Race { arms, .. } => { + for arm in arms { + collect_generic_callable_name_type_params_from_expr(&arm.awaitable, out); + collect_generic_callable_name_type_params_from_expr(&arm.body, out); + } + } + IrExprKind::Range { start, end, .. } => { + for expr in [start, end].into_iter().flatten() { + collect_generic_callable_name_type_params_from_expr(expr, out); + } + } + IrExprKind::Format { parts } => { + for part in parts { + if let super::super::super::expr::FormatPart::Expr { expr, .. } = part { + collect_generic_callable_name_type_params_from_expr(expr, out); + } + } + } + IrExprKind::Var { .. } + | IrExprKind::StaticRead { .. } + | IrExprKind::StaticBinding { .. } + | IrExprKind::AssociatedFunction { .. } + | IrExprKind::Unit + | IrExprKind::None + | IrExprKind::Bool(_) + | IrExprKind::Int(_) + | IrExprKind::IntLiteral(_) + | IrExprKind::Float(_) + | IrExprKind::Decimal(_) + | IrExprKind::String(_) + | IrExprKind::Bytes(_) + | IrExprKind::Literal(_) + | IrExprKind::FieldsList(_) + | IrExprKind::SerdeToJson + | IrExprKind::SerdeFromJson(_) => {} + } +} + +fn collect_generic_callable_name_type_params_from_stmts(stmts: &[IrStmt], out: &mut Vec) { + for stmt in stmts { + match &stmt.kind { + IrStmtKind::Expr(expr) + | IrStmtKind::Yield(expr) + | IrStmtKind::Let { value: expr, .. } + | IrStmtKind::CompoundAssign { value: expr, .. } => { + collect_generic_callable_name_type_params_from_expr(expr, out); + } + IrStmtKind::Assign { target, value } => { + collect_generic_callable_name_type_params_from_assign_target(target, out); + collect_generic_callable_name_type_params_from_expr(value, out); + } + IrStmtKind::Return(Some(expr)) => collect_generic_callable_name_type_params_from_expr(expr, out), + IrStmtKind::Break { value: Some(expr), .. } => { + collect_generic_callable_name_type_params_from_expr(expr, out); + } + IrStmtKind::While { condition, body, .. } => { + collect_generic_callable_name_type_params_from_expr(condition, out); + collect_generic_callable_name_type_params_from_stmts(body, out); + } + IrStmtKind::For { iterable, body, .. } => { + collect_generic_callable_name_type_params_from_expr(iterable, out); + collect_generic_callable_name_type_params_from_stmts(body, out); + } + IrStmtKind::Loop { body, .. } | IrStmtKind::Block(body) => { + collect_generic_callable_name_type_params_from_stmts(body, out); + } + IrStmtKind::If { + condition, + then_branch, + else_branch, + } => { + collect_generic_callable_name_type_params_from_expr(condition, out); + collect_generic_callable_name_type_params_from_stmts(then_branch, out); + if let Some(else_branch) = else_branch { + collect_generic_callable_name_type_params_from_stmts(else_branch, out); + } + } + IrStmtKind::Match { scrutinee, arms } => { + collect_generic_callable_name_type_params_from_expr(scrutinee, out); + for arm in arms { + if let Some(guard) = &arm.guard { + collect_generic_callable_name_type_params_from_expr(guard, out); + } + collect_generic_callable_name_type_params_from_expr(&arm.body, out); + } + } + IrStmtKind::Return(None) | IrStmtKind::Break { value: None, .. } | IrStmtKind::Continue(_) => {} + } + } +} + +fn collect_generic_callable_name_type_params_from_assign_target(target: &AssignTarget, out: &mut Vec) { + match target { + AssignTarget::Field { object, .. } => collect_generic_callable_name_type_params_from_expr(object, out), + AssignTarget::Index { object, index } => { + collect_generic_callable_name_type_params_from_expr(object, out); + collect_generic_callable_name_type_params_from_expr(index, out); + } + AssignTarget::Var(_) | AssignTarget::StaticBinding(_) | AssignTarget::Static(_) => {} + } +} + impl AstLowering { /// Lower a function declaration. /// @@ -133,6 +398,21 @@ impl AstLowering { let mut all_type_params = Self::lower_type_params(&f.type_params); all_type_params.extend(hidden_type_params); + let mut callable_name_type_params = Vec::new(); + collect_generic_callable_name_type_params_from_stmts(&body, &mut callable_name_type_params); + for type_param_name in callable_name_type_params { + if let Some(type_param) = all_type_params + .iter_mut() + .find(|type_param| type_param.name == type_param_name) + && !type_param.bounds.iter().any(|bound| { + bound.trait_path == "__IncanCallableName" + && bound.type_args.is_empty() + && bound.assoc_types.is_empty() + }) + { + type_param.bounds.push(IrTraitBound::simple("__IncanCallableName")); + } + } if is_generator { for type_param in &mut all_type_params { for trait_path in ["Send", "Static"] { @@ -175,6 +455,15 @@ impl AstLowering { format!("__incan_decorated_{name}") } + /// Return the span used for synthetic decorator callee nodes. + /// + /// The full decorator factory call keeps the source decorator span for typechecker handoff. Nested synthetic + /// callees must not reuse that span because expression metadata is span-keyed and the factory result type would + /// otherwise overwrite the callee's callable signature during lowering. + pub(in crate::backend::ir::lower) fn decorator_synthetic_callee_span() -> ast::Span { + ast::Span::default() + } + /// Build an expression that resolves a decorator's path through ordinary expression lowering. pub(in crate::backend::ir::lower) fn decorator_path_expr( decorator: &ast::Decorator, @@ -220,22 +509,26 @@ impl AstLowering { is_absolute: decorator.node.path.is_absolute, segments: path[..path.len() - 1].to_vec(), }; - let base = Self::decorator_path_expr_from_import_path(&base_path, decorator.span); + let base = + Self::decorator_path_expr_from_import_path(&base_path, Self::decorator_synthetic_callee_span()); let method = path.last().cloned().unwrap_or_default(); Spanned::new( - Expr::MethodCall(Box::new(base), method, Vec::new(), args), + Expr::MethodCall(Box::new(base), method, decorator.node.type_args.clone(), args), decorator.span, ) } else { - let callee = Self::decorator_path_expr(&decorator.node, decorator.span); - Spanned::new(Expr::Call(Box::new(callee), Vec::new(), args), decorator.span) + let callee = Self::decorator_path_expr(&decorator.node, Self::decorator_synthetic_callee_span()); + Spanned::new( + Expr::Call(Box::new(callee), decorator.node.type_args.clone(), args), + decorator.span, + ) } } else { Self::decorator_path_expr(&decorator.node, decorator.span) }; current = Spanned::new( Expr::Call(Box::new(callable), Vec::new(), vec![ast::CallArg::Positional(current)]), - decorator.span, + Self::decorator_synthetic_callee_span(), ); } Ok(current) diff --git a/src/backend/ir/lower/decl/helpers.rs b/src/backend/ir/lower/decl/helpers.rs index 3a53ccbd4..c5b0b7405 100644 --- a/src/backend/ir/lower/decl/helpers.rs +++ b/src/backend/ir/lower/decl/helpers.rs @@ -585,6 +585,7 @@ impl AstLowering { parent_levels: 0, }, name: derive_name.to_string(), + type_args: Vec::new(), is_call: false, args: Vec::new(), }, diff --git a/src/backend/ir/lower/decl/imports.rs b/src/backend/ir/lower/decl/imports.rs index cac4cad1f..e59a2cf82 100644 --- a/src/backend/ir/lower/decl/imports.rs +++ b/src/backend/ir/lower/decl/imports.rs @@ -84,6 +84,10 @@ impl AstLowering { super::super::super::decl::IrImportItem { name: item.name.clone(), alias: item.alias.clone(), + is_static: self + .type_info + .as_ref() + .is_some_and(|info| info.static_binding(binding_name).is_some()), rust_trait_import, } }) diff --git a/src/backend/ir/lower/decl/methods.rs b/src/backend/ir/lower/decl/methods.rs index 3f3544b8b..f65960751 100644 --- a/src/backend/ir/lower/decl/methods.rs +++ b/src/backend/ir/lower/decl/methods.rs @@ -73,15 +73,19 @@ impl AstLowering { is_absolute: decorator.node.path.is_absolute, segments: path[..path.len() - 1].to_vec(), }; - let base = Self::decorator_path_expr_from_import_path(&base_path, decorator.span); + let base = + Self::decorator_path_expr_from_import_path(&base_path, Self::decorator_synthetic_callee_span()); let method_name = path.last().cloned().unwrap_or_default(); Spanned::new( - ast::Expr::MethodCall(Box::new(base), method_name, Vec::new(), args), + ast::Expr::MethodCall(Box::new(base), method_name, decorator.node.type_args.clone(), args), decorator.span, ) } else { - let callee = Self::decorator_path_expr(&decorator.node, decorator.span); - Spanned::new(ast::Expr::Call(Box::new(callee), Vec::new(), args), decorator.span) + let callee = Self::decorator_path_expr(&decorator.node, Self::decorator_synthetic_callee_span()); + Spanned::new( + ast::Expr::Call(Box::new(callee), decorator.node.type_args.clone(), args), + decorator.span, + ) } } else { Self::decorator_path_expr(&decorator.node, decorator.span) @@ -93,7 +97,7 @@ impl AstLowering { }; current = Spanned::new( ast::Expr::Call(Box::new(callable), Vec::new(), vec![ast::CallArg::Positional(arg)]), - decorator.span, + Self::decorator_synthetic_callee_span(), ); } Ok(current) @@ -350,7 +354,7 @@ impl AstLowering { type_param_names, )?; let adapter = self.decorated_method_original_adapter(owner, method)?; - let wrapper = self.lower_decorated_method_wrapper(owner, method)?; + let wrapper = self.lower_decorated_method_wrapper(owner, method, type_param_names)?; Ok(vec![original, adapter, wrapper]) } else { Ok(vec![self.lower_method_with_type_params(method, type_param_names)?]) @@ -362,6 +366,7 @@ impl AstLowering { &mut self, owner: &str, method: &ast::MethodDecl, + owner_type_param_names: Option<&HashSet<&str>>, ) -> Result { let Some(binding) = self.type_info.as_ref().and_then(|info| { info.declarations @@ -369,15 +374,23 @@ impl AstLowering { .get(&(owner.to_string(), method.name.clone())) .cloned() }) else { - return self.lower_method_with_type_params(method, None); + return self.lower_method_with_type_params(method, owner_type_param_names); }; let crate::frontend::symbols::ResolvedType::Function(params, ret) = binding.unbound_ty else { - return self.lower_method_with_type_params(method, None); + return self.lower_method_with_type_params(method, owner_type_param_names); }; let Some((receiver_param, surface_params)) = params.split_first() else { - return self.lower_method_with_type_params(method, None); + return self.lower_method_with_type_params(method, owner_type_param_names); }; let receiver_ty = self.lower_resolved_type(&receiver_param.ty); + let original_surface_params = match binding.original_unbound_ty { + crate::frontend::symbols::ResolvedType::Function(original_params, _) => { + original_params.into_iter().skip(1).collect::>() + } + _ => Vec::new(), + }; + let defaults = + self.decorated_param_defaults_for_surface(surface_params, &original_surface_params, &method.params); let mut wrapper_params = Vec::with_capacity(surface_params.len() + 1); let receiver = method.receiver.unwrap_or(ast::Receiver::Immutable); wrapper_params.push(FunctionParam { @@ -400,7 +413,7 @@ impl AstLowering { mutability: Mutability::Immutable, is_self: false, kind: param.kind, - default: None, + default: defaults.get(idx).cloned().flatten(), } })); let return_type = self.lower_resolved_type(&ret); @@ -680,11 +693,16 @@ impl AstLowering { if let Some(trait_id) = core_traits::from_str(short_name) { return core_traits::method_names(trait_id); } - match short_name { - "Callable0" | "Callable1" | "Callable2" => &["__call__"], - "Serialize" | "JsonSerialize" => &["to_json"], - "Deserialize" | "JsonDeserialize" => &["from_json"], - _ => &[], + if matches!(short_name, "Callable0" | "Callable1" | "Callable2") { + &["__call__"] + } else { + match incan_core::lang::stdlib::stdlib_json_trait_id(trait_name) + .or_else(|| incan_core::lang::stdlib::stdlib_json_trait_id(short_name)) + { + Some(incan_core::lang::stdlib::StdlibJsonTraitId::Serialize) => &["to_json"], + Some(incan_core::lang::stdlib::StdlibJsonTraitId::Deserialize) => &["from_json"], + None => &[], + } } } @@ -698,13 +716,13 @@ impl AstLowering { .rsplit(['.', ':']) .find(|segment| !segment.is_empty()) .unwrap_or(trait_name); - matches!( - (short_name, method_name), - ("Serialize", "to_json") - | ("JsonSerialize", "to_json") - | ("Deserialize", "from_json") - | ("JsonDeserialize", "from_json") - ) + match incan_core::lang::stdlib::stdlib_json_trait_id(trait_name) + .or_else(|| incan_core::lang::stdlib::stdlib_json_trait_id(short_name)) + { + Some(incan_core::lang::stdlib::StdlibJsonTraitId::Serialize) => method_name == "to_json", + Some(incan_core::lang::stdlib::StdlibJsonTraitId::Deserialize) => method_name == "from_json", + None => false, + } } /// Return whether a method is safe to emit into an imported trait impl when the trait declaration is missing. diff --git a/src/backend/ir/lower/decl/mod.rs b/src/backend/ir/lower/decl/mod.rs index 1d1e77c66..9125e33b2 100644 --- a/src/backend/ir/lower/decl/mod.rs +++ b/src/backend/ir/lower/decl/mod.rs @@ -16,7 +16,10 @@ mod newtypes; mod traits; use super::super::IrSpan; -use super::super::decl::{IrDecl, IrDeclKind, IrInteropAdapterKind, IrInteropDirection, IrInteropEdge, Visibility}; +use super::super::decl::{ + IrDecl, IrDeclKind, IrImportOrigin, IrImportQualifier, IrInteropAdapterKind, IrInteropDirection, IrInteropEdge, + Visibility, +}; use super::super::types::IrType; use super::AstLowering; use super::errors::LoweringError; @@ -138,11 +141,16 @@ impl AstLowering { is_rusttype: false, interop_edges: Vec::new(), }, - ast::Declaration::Alias(a) => IrDeclKind::SymbolAlias { - visibility: Self::map_visibility(a.visibility), - name: a.name.clone(), - target_path: a.target.segments.clone(), - }, + ast::Declaration::Alias(a) => { + let (target_path, target_origin, target_qualifier) = self.alias_reexport_target(&a.target.segments); + IrDeclKind::SymbolAlias { + visibility: Self::map_visibility(a.visibility), + name: a.name.clone(), + target_path, + target_origin, + target_qualifier, + } + } ast::Declaration::Partial(_) => { return Err(LoweringError { message: "Partial callable presets are not lowered by this syntax-only slice".to_string(), @@ -188,6 +196,26 @@ impl AstLowering { Ok(IrDecl::new(kind)) } + /// Resolve the path that should be used when emitting a module-level alias declaration. + /// + /// A source alias can target a local import binding, but generated Rust public re-exports must point at the + /// imported item path itself. Expression lowering still keeps the source binding for ordinary calls. + fn alias_reexport_target( + &self, + segments: &[String], + ) -> (Vec, Option, Option) { + if let [target] = segments + && let Some(imported) = self.imported_alias_targets.get(target) + { + return ( + imported.path.clone(), + Some(imported.origin.clone()), + Some(imported.qualifier), + ); + } + (segments.to_vec(), None, None) + } + fn lower_interop_edges( &mut self, edges: &[ast::Spanned], diff --git a/src/backend/ir/lower/expr/calls.rs b/src/backend/ir/lower/expr/calls.rs index a79ca89c8..32d102605 100644 --- a/src/backend/ir/lower/expr/calls.rs +++ b/src/backend/ir/lower/expr/calls.rs @@ -20,6 +20,8 @@ use incan_core::lang::stdlib; use incan_core::lang::stdlib::{STDLIB_BUILTINS, STDLIB_ROOT}; use incan_core::lang::surface::constructors::{self, ConstructorId}; use incan_core::lang::surface::types as surface_types; +use incan_core::lang::testing::{self, TestingAssertHelperId}; +use incan_core::lang::types::collections::{self, CollectionTypeId}; const TYPE_CONSTRUCTOR_HOOK: &str = "__incan_new"; @@ -1204,25 +1206,6 @@ impl AstLowering { } } - /// Build a synthetic callable signature from an already-lowered function type. - fn function_signature_from_ir_type(params: &[IrType], ret: &IrType) -> FunctionSignature { - FunctionSignature { - params: params - .iter() - .enumerate() - .map(|(idx, ty)| FunctionParam { - name: format!("__incan_arg_{idx}"), - ty: ty.clone(), - mutability: super::super::super::types::Mutability::Immutable, - is_self: false, - kind: ast::ParamKind::Normal, - default: None, - }) - .collect(), - return_type: ret.clone(), - } - } - /// Return whether passing `arg` to a callable parameter should refine that parameter to a shared borrow. fn callable_arg_needs_implicit_borrow(arg: &TypedExpr, target_ty: &IrType) -> bool { if arg.ty.is_copy() || matches!(target_ty, IrType::Ref(_) | IrType::RefMut(_)) { @@ -1261,7 +1244,7 @@ impl AstLowering { return callable_signature; }; let mut signature = - callable_signature.unwrap_or_else(|| Self::function_signature_from_ir_type(params, ret.as_ref())); + callable_signature.unwrap_or_else(|| FunctionSignature::from_function_type(params, ret.as_ref())); let mut changed = false; for (idx, arg) in args.iter().enumerate() { @@ -1391,7 +1374,7 @@ impl AstLowering { let Some(coercion) = coercion else { return Ok(arg_expr); }; - let target_ty = self.lower_resolved_type(&coercion.target_type); + let target_ty = self.lower_rust_boundary_target_type(&coercion.target_type); let from_ty = arg_expr.ty.clone(); let kind = match coercion.kind { RustArgCoercionKind::Builtin(policy) => IrInteropCoercionKind::Builtin { @@ -1421,6 +1404,104 @@ impl AstLowering { )) } + /// Lower the typechecker-selected Rust boundary target without collapsing borrowed Rust slices into owned values. + /// + /// General source-level references lower as `Ref`, but Rust argument coercions use the target type as a backend + /// contract. A `&str` parameter therefore lowers to `StrRef`, while `&String` remains a reference to the owned Rust + /// string target recorded by the frontend. + fn lower_rust_boundary_target_type(&self, target_ty: &ResolvedType) -> IrType { + match target_ty { + ResolvedType::Ref(inner) if matches!(inner.as_ref(), ResolvedType::Str) => IrType::StrRef, + ResolvedType::Ref(inner) => IrType::Ref(Box::new(self.lower_rust_boundary_target_type(inner))), + ResolvedType::RefMut(inner) => IrType::RefMut(Box::new(self.lower_rust_boundary_target_type(inner))), + ResolvedType::Tuple(items) => IrType::Tuple( + items + .iter() + .map(|item| self.lower_rust_boundary_target_type(item)) + .collect(), + ), + ResolvedType::FrozenList(inner) => IrType::NamedGeneric( + collections::as_str(CollectionTypeId::FrozenList).to_string(), + vec![self.lower_rust_boundary_target_type(inner)], + ), + ResolvedType::FrozenSet(inner) => IrType::NamedGeneric( + collections::as_str(CollectionTypeId::FrozenSet).to_string(), + vec![self.lower_rust_boundary_target_type(inner)], + ), + ResolvedType::FrozenDict(key, value) => IrType::NamedGeneric( + collections::as_str(CollectionTypeId::FrozenDict).to_string(), + vec![ + self.lower_rust_boundary_target_type(key), + self.lower_rust_boundary_target_type(value), + ], + ), + ResolvedType::Generic(name, args) => match collections::from_str(name.as_str()) { + Some(CollectionTypeId::List) => IrType::List(Box::new( + args.first() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + )), + Some(CollectionTypeId::Dict) => IrType::Dict( + Box::new( + args.first() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + ), + Box::new( + args.get(1) + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + ), + ), + Some(CollectionTypeId::Set) => IrType::Set(Box::new( + args.first() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + )), + Some(CollectionTypeId::Option) => IrType::Option(Box::new( + args.first() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + )), + Some(CollectionTypeId::Result) => IrType::Result( + Box::new( + args.first() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + ), + Box::new( + args.get(1) + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .unwrap_or(IrType::Unknown), + ), + ), + Some(CollectionTypeId::Tuple) => IrType::Tuple( + args.iter() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .collect(), + ), + Some( + id @ (CollectionTypeId::FrozenList + | CollectionTypeId::FrozenSet + | CollectionTypeId::FrozenDict + | CollectionTypeId::Generator), + ) => IrType::NamedGeneric( + collections::as_str(id).to_string(), + args.iter() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .collect(), + ), + None => IrType::NamedGeneric( + name.clone(), + args.iter() + .map(|arg| self.lower_rust_boundary_target_type(arg)) + .collect(), + ), + }, + _ => self.lower_resolved_type(target_ty), + } + } + /// Lower a function/constructor call expression. /// /// Handles struct constructors, builtin functions, newtype checked construction, and regular function calls. @@ -1561,6 +1642,11 @@ impl AstLowering { if let ast::Expr::Ident(name) = &f.node && let Some(builtin) = BuiltinFn::from_name(name) && imported_callee_path.is_none() + && self + .type_info + .as_ref() + .is_none_or(|info| info.ident_kind(f.span).is_none()) + && self.callable_signature_for_call_span(call_span).is_none() && !matches!(func.ty, IrType::Function { .. }) { let args_ir = self.lower_call_args(args)?.into_iter().map(|a| a.expr).collect(); @@ -1595,11 +1681,12 @@ impl AstLowering { let arg_span = Self::call_arg_expr(arg_ast).span; arg_ir.expr = self.wrap_with_rust_arg_coercion(arg_ir.expr.clone(), arg_span)?; } - if imported_callee_path.as_ref().is_some_and(|path| { - path.len() == 3 && path[0] == "std" && path[1] == "testing" && path[2] == "assert_raises" - }) && args_ir - .get(1) - .is_none_or(|arg| !matches!(arg.expr.kind, IrExprKind::Literal(IrLiteral::StaticStr(_)))) + if imported_callee_path + .as_ref() + .is_some_and(|path| testing::is_assert_helper_std_path(path, TestingAssertHelperId::AssertRaises)) + && args_ir + .get(1) + .is_none_or(|arg| !matches!(arg.expr.kind, IrExprKind::Literal(IrLiteral::StaticStr(_)))) { let Some(error_type) = type_args.first() else { return Err(LoweringError { @@ -2097,7 +2184,7 @@ mod tests { (arg_span.start, arg_span.end), RustArgCoercionInfo { rust_target_type: "&str".to_string(), - target_type: ResolvedType::Str, + target_type: ResolvedType::Ref(Box::new(ResolvedType::Str)), kind: RustArgCoercionKind::Builtin(CoercionPolicy::Borrow), }, ); @@ -2119,19 +2206,40 @@ mod tests { match lowered.kind { IrExprKind::MethodCall { args, .. } => { - assert!( - matches!( - args.first().map(|arg| &arg.expr.kind), - Some(IrExprKind::InteropCoerce { .. }) - ), - "expected first method arg to be wrapped in InteropCoerce, got {args:?}" - ); + let Some(first_arg) = args.first() else { + return Err("expected lowered method arg".to_string()); + }; + match &first_arg.expr.kind { + IrExprKind::InteropCoerce { to_ty, .. } => { + assert_eq!( + *to_ty, + IrType::StrRef, + "expected borrowed str target to lower to StrRef" + ); + } + other => { + return Err(format!( + "expected first method arg to be wrapped in InteropCoerce, got {other:?}" + )); + } + } } other => return Err(format!("expected MethodCall lowering, got {other:?}")), } Ok(()) } + #[test] + fn lower_rust_boundary_target_preserves_nested_borrowed_str_refs() { + let lowering = AstLowering::new(); + let target = ResolvedType::Generic("List".to_string(), vec![ResolvedType::Ref(Box::new(ResolvedType::Str))]); + + assert_eq!( + lowering.lower_rust_boundary_target_type(&target), + IrType::List(Box::new(IrType::StrRef)), + ); + } + #[test] fn lower_method_call_threads_arg_shape_hint_from_typechecker() -> Result<(), String> { let receiver_span = Span::new(0, 5); diff --git a/src/backend/ir/lower/expr/mod.rs b/src/backend/ir/lower/expr/mod.rs index 4ac449765..733bf9c19 100644 --- a/src/backend/ir/lower/expr/mod.rs +++ b/src/backend/ir/lower/expr/mod.rs @@ -370,49 +370,67 @@ impl AstLowering { /// This is a stepping stone toward fully typed lowering. pub fn lower_expr_spanned(&mut self, expr: &Spanned) -> Result { let mut lowered = self.lower_expr(&expr.node, expr.span)?; - if let Some(info) = &self.type_info { - if let Some(res_ty) = info.expr_type(expr.span) { - // Preserve reference wrappers introduced by lowering (e.g. mutable parameters are tracked as - // `RefMut(T)` in IR), while still benefiting from the typechecker's inner type information. - // - // The frontend type system does not model references, so `expr_type` typically returns `T` where - // lowering may have already marked the same binding as `Ref(T)`/`RefMut(T)`. - // - // Likewise, RFC-008 const lowering may have already refined `str`/`bytes` to their static IR forms. - // Keep those backend-specific const representations intact so later emission can materialize owned - // values only when required. - let inferred = self.lower_resolved_type(res_ty); - lowered.ty = match &lowered.ty { - IrType::Ref(existing_inner) => { - IrType::Ref(Box::new(Self::merge_inferred_ir_type(existing_inner, inferred))) - } - IrType::RefMut(existing_inner) => { - IrType::RefMut(Box::new(Self::merge_inferred_ir_type(existing_inner, inferred))) - } - IrType::StaticStr => IrType::StaticStr, - IrType::StaticBytes => IrType::StaticBytes, - existing => Self::merge_inferred_ir_type(existing, inferred), - }; - } - if let Some(kind) = info.ident_kind(expr.span) { - match (&expr.node, &mut lowered.kind) { - (ast::Expr::Ident(name), _) if matches!(kind, IdentKind::Static) => { - lowered.kind = IrExprKind::StaticRead { name: name.clone() }; - } - (_, IrExprKind::Var { ref_kind, .. }) => { - *ref_kind = match kind { - IdentKind::Value => *ref_kind, - IdentKind::Static => *ref_kind, - IdentKind::TypeName => VarRefKind::TypeName, - IdentKind::Variant => VarRefKind::TypeName, - IdentKind::Module => VarRefKind::ExternalName, - IdentKind::RustImport => VarRefKind::ExternalRustName, - IdentKind::RustValue => VarRefKind::Value, - IdentKind::Trait => VarRefKind::TypeName, - }; + if let Some(info) = &self.type_info + && let Some(res_ty) = info.expr_type(expr.span) + { + // Preserve reference wrappers introduced by lowering (e.g. mutable parameters are tracked as + // `RefMut(T)` in IR), while still benefiting from the typechecker's inner type information. + // + // The frontend type system does not model references, so `expr_type` typically returns `T` where + // lowering may have already marked the same binding as `Ref(T)`/`RefMut(T)`. + // + // Likewise, RFC-008 const lowering may have already refined `str`/`bytes` to their static IR forms. + // Keep those backend-specific const representations intact so later emission can materialize owned + // values only when required. + let inferred = self.lower_resolved_type(res_ty); + lowered.ty = match &lowered.ty { + IrType::Ref(existing_inner) => { + IrType::Ref(Box::new(Self::merge_inferred_ir_type(existing_inner, inferred))) + } + IrType::RefMut(existing_inner) => { + IrType::RefMut(Box::new(Self::merge_inferred_ir_type(existing_inner, inferred))) + } + IrType::StaticStr => IrType::StaticStr, + IrType::StaticBytes => IrType::StaticBytes, + existing => Self::merge_inferred_ir_type(existing, inferred), + }; + } + if let Some(kind) = self.ident_kind_for_lowering(expr) { + match (&expr.node, &mut lowered.kind) { + (ast::Expr::Ident(name), _) if matches!(kind, IdentKind::Static) => { + lowered.kind = IrExprKind::StaticRead { name: name.clone() }; + } + (ast::Expr::Ident(name), IrExprKind::Var { ref_kind, .. }) => { + *ref_kind = match kind { + IdentKind::Value => *ref_kind, + IdentKind::Static => *ref_kind, + IdentKind::TypeName => VarRefKind::TypeName, + IdentKind::Variant => VarRefKind::TypeName, + IdentKind::Module => VarRefKind::ExternalName, + IdentKind::RustImport => VarRefKind::ExternalRustName, + IdentKind::RustValue => VarRefKind::Value, + IdentKind::Trait => VarRefKind::TypeName, + }; + if matches!(kind, IdentKind::TypeName | IdentKind::Variant | IdentKind::Trait) + && matches!(lowered.ty, IrType::Unknown) + && let Some(ty) = self.synthetic_type_ident_ir_type(name) + { + lowered.ty = ty; } - _ => {} } + (_, IrExprKind::Var { ref_kind, .. }) => { + *ref_kind = match kind { + IdentKind::Value => *ref_kind, + IdentKind::Static => *ref_kind, + IdentKind::TypeName => VarRefKind::TypeName, + IdentKind::Variant => VarRefKind::TypeName, + IdentKind::Module => VarRefKind::ExternalName, + IdentKind::RustImport => VarRefKind::ExternalRustName, + IdentKind::RustValue => VarRefKind::Value, + IdentKind::Trait => VarRefKind::TypeName, + }; + } + _ => {} } } // Apply any rusttype method return coercion recorded by the typechecker (e.g. &str → String). @@ -422,6 +440,53 @@ impl AstLowering { Ok(lowered) } + /// Return the identifier classification that lowering should use for this expression. + /// + /// Most source expressions use span-keyed frontend metadata. Synthetic expressions created by lowering, such as + /// user-defined decorator factory calls, intentionally use the default span so they do not collide with call-site + /// expression types. Those synthetic nodes still need metadata-backed classification for type names and module + /// statics; otherwise they fall back to value-shaped Rust emission. + fn ident_kind_for_lowering(&self, expr: &Spanned) -> Option { + if let Some(kind) = self.type_info.as_ref().and_then(|info| info.ident_kind(expr.span)) { + return Some(kind); + } + if expr.span != ast::Span::default() { + return None; + } + let ast::Expr::Ident(name) = &expr.node else { + return None; + }; + if self + .type_info + .as_ref() + .is_some_and(|info| info.static_binding(name).is_some()) + { + return Some(IdentKind::Static); + } + if self.synthetic_type_ident_ir_type(name).is_some() { + return Some(IdentKind::TypeName); + } + None + } + + /// Return the known IR type for a synthetic type-like identifier. + fn synthetic_type_ident_ir_type(&self, name: &str) -> Option { + self.struct_names + .get(name) + .cloned() + .or_else(|| self.enum_names.get(name).cloned()) + .or_else(|| { + self.class_decls + .contains_key(name) + .then(|| IrType::Struct(name.to_string())) + }) + .or_else(|| { + self.trait_decls + .contains_key(name) + .then(|| IrType::Struct(name.to_string())) + }) + } + /// Lower an expression to IR. /// /// Handles all expression types including: @@ -1259,9 +1324,13 @@ impl AstLowering { .iter() .map(|part| match part { ast::FStringPart::Literal(s) => Ok(super::super::expr::FormatPart::Literal(s.clone())), - ast::FStringPart::Expr(e) => { - let lowered = self.lower_expr_spanned(e)?; - Ok(super::super::expr::FormatPart::Expr(lowered)) + ast::FStringPart::Expr { expr, format } => { + let lowered = self.lower_expr_spanned(expr)?; + let style = match format { + ast::FStringFormat::Display => super::super::expr::FormatStyle::Display, + ast::FStringFormat::Debug => super::super::expr::FormatStyle::Debug, + }; + Ok(super::super::expr::FormatPart::Expr { expr: lowered, style }) } }) .collect::, LoweringError>>()?; diff --git a/src/backend/ir/lower/mod.rs b/src/backend/ir/lower/mod.rs index 98f70f752..698f67a4b 100644 --- a/src/backend/ir/lower/mod.rs +++ b/src/backend/ir/lower/mod.rs @@ -35,14 +35,14 @@ mod types; use std::collections::{HashMap, HashSet}; use super::TypedExpr; -use super::decl::{FunctionParam, IrDecl, IrDeclKind}; +use super::decl::{FunctionParam, IrDecl, IrDeclKind, IrImportOrigin, IrImportQualifier}; use super::expr::{IrCallArg, IrCallArgKind, IrExprKind, VarAccess, VarRefKind}; use super::stmt::{IrStmt, IrStmtKind}; use super::types::IrType; -use super::{FunctionSignature, IrProgram, Mutability}; +use super::{FunctionReexport, FunctionSignature, IrProgram, Mutability}; use crate::frontend::ast; use crate::frontend::decorator_resolution; -use crate::frontend::symbols::NewtypePrimitiveConstraint; +use crate::frontend::symbols::{CallableParam, NewtypePrimitiveConstraint}; use crate::frontend::typechecker::TypeCheckInfo; use crate::frontend::typechecker::stdlib_loader::StdlibAstCache; use incan_core::lang::conventions; @@ -64,6 +64,13 @@ pub(in crate::backend::ir::lower) struct TraitImplLoweringInput<'a> { pub impl_associated_types: &'a [ast::Spanned], } +#[derive(Debug, Clone)] +pub(super) struct ImportedAliasTarget { + pub origin: IrImportOrigin, + pub qualifier: IrImportQualifier, + pub path: Vec, +} + /// AST to IR lowering context. /// /// Maintains state needed during the lowering pass: @@ -144,6 +151,8 @@ pub struct AstLowering { pub(super) callable_param_scopes: Vec>, /// Module-level symbol aliases mapped from alias name to canonical target name. pub(super) symbol_aliases: HashMap, + /// Imported item bindings mapped to their original import paths for public alias re-export emission. + pub(super) imported_alias_targets: HashMap, /// Cached stdlib metadata used to resolve rust.module-backed decorators/derives. pub(super) stdlib_cache: StdlibAstCache, /// `rusttype` underlying Rust type lookup by alias name. @@ -235,6 +244,7 @@ impl AstLowering { rust_import_aliases: HashMap::new(), callable_param_scopes: Vec::new(), symbol_aliases: HashMap::new(), + imported_alias_targets: HashMap::new(), stdlib_cache: StdlibAstCache::new(), rusttype_underlying: HashMap::new(), rusttype_interop_edges: HashMap::new(), @@ -248,6 +258,68 @@ impl AstLowering { self.current_source_module_name = name; } + /// Lower one typechecker-resolved callable surface into IR parameters, attaching an already-planned default + /// expression for each parameter when present. + fn function_params_from_callable_surface( + &mut self, + callable_params: &[CallableParam], + defaults: &[Option], + ) -> Vec { + callable_params + .iter() + .enumerate() + .map(|(idx, param)| { + let base_ty = self.lower_resolved_type(¶m.ty); + FunctionParam { + name: param.name.clone().unwrap_or_else(|| format!("__incan_arg_{idx}")), + ty: Self::lower_param_container_type(param.kind, base_ty), + mutability: Mutability::Immutable, + is_self: false, + kind: param.kind, + default: defaults.get(idx).cloned().flatten(), + } + }) + .collect() + } + + fn function_params_from_source_callable_surface( + &mut self, + callable_params: &[CallableParam], + source_params: &[ast::Spanned], + ) -> Vec { + callable_params + .iter() + .enumerate() + .map(|(idx, param)| { + let source_idx = param + .name + .as_deref() + .and_then(|name| source_params.iter().position(|source| source.node.name == name)) + .unwrap_or(idx); + let source_param = source_params.get(source_idx); + let default = if param.has_default { + source_param + .and_then(|source| source.node.default.as_ref()) + .and_then(|default_expr| self.lower_expr_spanned(default_expr).ok()) + } else { + None + }; + FunctionParam { + name: param.name.clone().unwrap_or_else(|| format!("__incan_arg_{idx}")), + ty: Self::lower_param_container_type(param.kind, self.lower_resolved_type(¶m.ty)), + mutability: if source_param.is_some_and(|source| source.node.is_mut) { + Mutability::Mutable + } else { + Mutability::Immutable + }, + is_self: false, + kind: param.kind, + default, + } + }) + .collect() + } + /// Return the logger name supplied to default `std.logging.get_logger()` calls. pub(super) fn current_default_logger_name(&self) -> String { self.current_source_module_name @@ -885,6 +957,50 @@ impl AstLowering { } } + fn collect_function_reexports(&self, program: &ast::Program) -> Vec { + let mut reexports = Vec::new(); + for decl in &program.declarations { + let ast::Declaration::Import(import) = &decl.node else { + continue; + }; + if !matches!(import.visibility, ast::Visibility::Public) { + continue; + } + let ast::ImportKind::From { module, items } = &import.kind else { + continue; + }; + + let module_path = self.canonical_source_import_module_segments(module); + for item in items { + let mut target_path = module_path.clone(); + target_path.push(item.name.clone()); + reexports.push(FunctionReexport { + name: item.alias.as_ref().unwrap_or(&item.name).clone(), + target_path, + }); + } + } + reexports + } + + fn canonical_source_import_module_segments(&self, module: &ast::ImportPath) -> Vec { + let segments = if module.parent_levels > 0 && !module.is_absolute { + let mut base = self + .current_source_module_name + .as_deref() + .map(|module_name| module_name.split('.').map(str::to_string).collect::>()) + .unwrap_or_default(); + for _ in 0..module.parent_levels { + base.pop(); + } + base.extend(module.segments.iter().cloned()); + base + } else { + module.segments.clone() + }; + crate::frontend::module::canonicalize_source_module_segments(&segments) + } + /// Lower a complete AST program to IR. /// /// This is the main entry point for the lowering pass. It performs: @@ -912,6 +1028,8 @@ impl AstLowering { let mut errors: Vec = Vec::new(); self.import_aliases = decorator_resolution::collect_import_aliases(program); self.rust_import_aliases = decorator_resolution::collect_rust_import_aliases(program); + ir_program.function_reexports = self.collect_function_reexports(program); + self.imported_alias_targets = self.collect_imported_alias_targets(program); self.seed_imported_stdlib_trait_decls(program); self.alias_imported_dependency_trait_decls(); self.symbol_aliases = program @@ -1050,55 +1168,78 @@ impl AstLowering { if let ast::Declaration::Function(ref f) = decl.node { let type_param_names: std::collections::HashSet<&str> = f.type_params.iter().map(|tp| tp.name.as_str()).collect(); - let params: Vec = f - .params - .iter() - .map(|p| { - let base_ty = self.lower_type_with_type_params(&p.node.ty.node, Some(&type_param_names)); - let param_ty = Self::lower_param_container_type(p.node.kind, base_ty); - FunctionParam { - name: p.node.name.clone(), - ty: param_ty, - mutability: if p.node.is_mut { - Mutability::Mutable - } else { - Mutability::Immutable - }, - is_self: false, - kind: p.node.kind, - default: match &p.node.default { - Some(default_expr) => self.lower_expr_spanned(default_expr).ok(), - None => None, - }, - } - }) - .collect(); - let return_type = self + let function_binding = self .type_info .as_ref() - .and_then(|info| info.declarations.decorated_function_bindings.get(&f.name)) - .and_then(|binding| match &binding.ty { - crate::frontend::symbols::ResolvedType::Function(_, ret) => Some(self.lower_resolved_type(ret)), - _ => None, - }) - .unwrap_or_else(|| self.lower_type_with_type_params(&f.return_type.node, Some(&type_param_names))); - ir_program - .function_registry - .register(f.name.clone(), params.clone(), return_type.clone()); - if let Some(signature) = ir_program.function_registry.get(&f.name).cloned() { - self.update_root_function_binding(&f.name, &signature.params, &signature.return_type); - } - if self + .and_then(|info| info.declarations.function_bindings.get(&f.name).cloned()); + let source_params: Vec = function_binding + .as_ref() + .map(|binding| self.function_params_from_source_callable_surface(&binding.params, &f.params)) + .unwrap_or_else(|| { + f.params + .iter() + .map(|p| { + let base_ty = + self.lower_type_with_type_params(&p.node.ty.node, Some(&type_param_names)); + let param_ty = Self::lower_param_container_type(p.node.kind, base_ty); + FunctionParam { + name: p.node.name.clone(), + ty: param_ty, + mutability: if p.node.is_mut { + Mutability::Mutable + } else { + Mutability::Immutable + }, + is_self: false, + kind: p.node.kind, + default: match &p.node.default { + Some(default_expr) => self.lower_expr_spanned(default_expr).ok(), + None => None, + }, + } + }) + .collect() + }); + if let Some(binding) = self .type_info .as_ref() - .is_some_and(|info| info.declarations.decorated_function_bindings.contains_key(&f.name)) + .and_then(|info| info.declarations.decorated_function_bindings.get(&f.name).cloned()) + && let crate::frontend::symbols::ResolvedType::Function(callable_params, callable_ret) = binding.ty { + let original_params = match &binding.original_ty { + crate::frontend::symbols::ResolvedType::Function(params, _) => params.as_slice(), + _ => &[], + }; + let defaults = + self.decorated_param_defaults_for_surface(&callable_params, original_params, &f.params); + let params = self.function_params_from_callable_surface(&callable_params, &defaults); + let return_type = self.lower_resolved_type(&callable_ret); + ir_program + .function_registry + .register(f.name.clone(), params.clone(), return_type.clone()); + self.update_root_function_binding(&f.name, ¶ms, &return_type); + let original_name = Self::decorator_original_function_name(&f.name); - let original_return_type = - self.lower_type_with_type_params(&f.return_type.node, Some(&type_param_names)); + let original_return_type = function_binding + .as_ref() + .map(|binding| self.lower_resolved_type(&binding.return_type)) + .unwrap_or_else(|| { + self.lower_type_with_type_params(&f.return_type.node, Some(&type_param_names)) + }); ir_program .function_registry - .register(original_name, params, original_return_type); + .register(original_name, source_params, original_return_type); + continue; + } + let return_type = function_binding + .as_ref() + .map(|binding| self.lower_resolved_type(&binding.return_type)) + .unwrap_or_else(|| self.lower_type_with_type_params(&f.return_type.node, Some(&type_param_names))); + ir_program + .function_registry + .register(f.name.clone(), source_params.clone(), return_type.clone()); + if let Some(signature) = ir_program.function_registry.get(&f.name).cloned() { + self.update_root_function_binding(&f.name, &signature.params, &signature.return_type); } } else if let ast::Declaration::Alias(ref alias) = decl.node && let [target] = alias.target.segments.as_slice() @@ -1536,6 +1677,40 @@ impl AstLowering { } } + /// Collect imported item bindings that module-level symbol aliases may need to re-export directly. + fn collect_imported_alias_targets(&self, program: &ast::Program) -> HashMap { + let mut targets = HashMap::new(); + for decl in &program.declarations { + let ast::Declaration::Import(import) = &decl.node else { + continue; + }; + let IrDeclKind::Import { + origin, + qualifier, + path, + items, + .. + } = self.lower_import(import) + else { + continue; + }; + for item in items { + let binding = item.alias.unwrap_or_else(|| item.name.clone()); + let mut item_path = path.clone(); + item_path.push(item.name); + targets.insert( + binding, + ImportedAliasTarget { + origin: origin.clone(), + qualifier, + path: item_path, + }, + ); + } + } + targets + } + /// Lower a function declaration, expanding RFC 036 decorated functions into original/static/wrapper items. fn lower_decorated_function_declarations(&mut self, f: &ast::FunctionDecl) -> Result, LoweringError> { let Some(binding) = self @@ -1554,6 +1729,10 @@ impl AstLowering { span: ast::Span::default().into(), }); }; + let original_params = match binding.original_ty { + crate::frontend::symbols::ResolvedType::Function(params, _) => params, + _ => Vec::new(), + }; let original_name = Self::decorator_original_function_name(&f.name); let original = self.lower_function_named(f, original_name.clone(), super::decl::Visibility::Private)?; @@ -1569,7 +1748,13 @@ impl AstLowering { let mut value = self.lower_expr_spanned(&decorator_expr)?; value.ty = decorated_ty.clone(); let static_name = Self::decorator_static_binding_name(&f.name); - let wrapper = self.decorated_function_wrapper(f, &static_name, &callable_params, callable_ret.as_ref()); + let wrapper = self.decorated_function_wrapper( + f, + &static_name, + &callable_params, + &original_params, + callable_ret.as_ref(), + ); Ok(vec![ IrDecl::new(IrDeclKind::Function(original)), @@ -1588,24 +1773,12 @@ impl AstLowering { &mut self, f: &ast::FunctionDecl, static_name: &str, - callable_params: &[crate::frontend::symbols::CallableParam], + callable_params: &[CallableParam], + original_params: &[CallableParam], callable_ret: &crate::frontend::symbols::ResolvedType, ) -> super::decl::IrFunction { - let params: Vec = callable_params - .iter() - .enumerate() - .map(|(idx, param)| { - let base_ty = self.lower_resolved_type(¶m.ty); - FunctionParam { - name: param.name.clone().unwrap_or_else(|| format!("__incan_arg_{idx}")), - ty: Self::lower_param_container_type(param.kind, base_ty), - mutability: Mutability::Immutable, - is_self: false, - kind: param.kind, - default: None, - } - }) - .collect(); + let defaults = self.decorated_param_defaults_for_surface(callable_params, original_params, &f.params); + let params = self.function_params_from_callable_surface(callable_params, &defaults); let return_type = self.lower_resolved_type(callable_ret); let static_func = TypedExpr::new( IrExprKind::StaticRead { @@ -1657,6 +1830,75 @@ impl AstLowering { } } + /// Lower source defaults for a decorated callable wrapper when the final callable surface still maps to the + /// original typechecker-resolved parameters. + /// + /// Function types can describe parameter types but not default expressions. User-defined decorators often return an + /// explicit function type such as `(int) -> int`, which erases the declaration's richer call-site defaults even + /// when the decorator keeps the same callable surface. This helper rebuilds one default plan from source parameter + /// metadata only after the final decorator surface still matches the original callable shape. The comparison uses + /// typechecker-resolved parameter types so transparent aliases like `type Expr = Union[...]` do not split lowering + /// behavior across import or alias boundaries. + pub(super) fn decorated_param_defaults_for_surface( + &mut self, + surface_params: &[CallableParam], + original_params: &[CallableParam], + source_params: &[ast::Spanned], + ) -> Vec> { + let positional_shapes_match = Self::decorated_positional_param_shapes_match(surface_params, original_params); + + surface_params + .iter() + .enumerate() + .map(|(idx, surface_param)| { + let default_expr = if let Some(name) = surface_param.name.as_deref() { + original_params + .iter() + .position(|original_param| { + original_param.name.as_deref() == Some(name) + && Self::decorated_param_shape_matches(surface_param, original_param) + }) + .and_then(|source_idx| { + original_params + .get(source_idx) + .is_some_and(|original_param| original_param.has_default) + .then(|| source_params.get(source_idx)) + .flatten() + }) + .and_then(|source_param| source_param.node.default.clone()) + } else if positional_shapes_match { + original_params + .get(idx) + .is_some_and(|original_param| original_param.has_default) + .then(|| source_params.get(idx)) + .flatten() + .and_then(|source_param| source_param.node.default.clone()) + } else { + None + }; + + default_expr.and_then(|expr| self.lower_expr_spanned(&expr).ok()) + }) + .collect() + } + + fn decorated_positional_param_shapes_match( + surface_params: &[CallableParam], + original_params: &[CallableParam], + ) -> bool { + surface_params.len() == original_params.len() + && surface_params + .iter() + .zip(original_params) + .all(|(surface_param, original_param)| { + Self::decorated_param_shape_matches(surface_param, original_param) + }) + } + + fn decorated_param_shape_matches(surface_param: &CallableParam, original_param: &CallableParam) -> bool { + surface_param.kind == original_param.kind && surface_param.ty == original_param.ty + } + /// Add alias-qualified dependency trait declarations so default methods can expand for imported derive aliases. fn alias_imported_dependency_trait_decls(&mut self) { let existing = self.trait_decls.clone(); diff --git a/src/backend/ir/lower/stmt.rs b/src/backend/ir/lower/stmt.rs index f8b00fc99..c12f5995a 100644 --- a/src/backend/ir/lower/stmt.rs +++ b/src/backend/ir/lower/stmt.rs @@ -2188,7 +2188,7 @@ impl AstLowering { ast::Expr::Constructor(_, args) => self.count_call_args_ident_reads(args, counts), ast::Expr::FString(parts) => { for part in parts { - if let ast::FStringPart::Expr(expr) = part { + if let ast::FStringPart::Expr { expr, .. } = part { self.count_expr_ident_reads(&expr.node, counts); } } diff --git a/src/backend/ir/mod.rs b/src/backend/ir/mod.rs index 1e9af8614..fb5ce1131 100644 --- a/src/backend/ir/mod.rs +++ b/src/backend/ir/mod.rs @@ -23,6 +23,7 @@ pub mod conversions; pub mod ownership; pub mod prelude; +pub(crate) mod reference_shape; pub mod codegen; pub mod decl; @@ -58,6 +59,84 @@ pub struct FunctionSignature { pub return_type: IrType, } +impl FunctionSignature { + /// Build a positional callable signature from a lowered function type. + pub fn from_function_type(params: &[IrType], ret: &IrType) -> Self { + Self { + params: params + .iter() + .enumerate() + .map(|(idx, ty)| FunctionParam { + name: format!("__incan_arg_{idx}"), + ty: ty.clone(), + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }) + .collect(), + return_type: ret.clone(), + } + } + + /// Return the effective call signature when one source carries precise callable type metadata and another carries + /// source defaults for the same callable surface. + pub fn merge_default_source( + primary: Option<&FunctionSignature>, + default_source: Option<&FunctionSignature>, + ) -> Option { + Self::merge_default_source_by(primary, default_source, |left, right| left == right) + } + + /// Return the effective call signature using a caller-supplied type equivalence rule for default inheritance. + pub fn merge_default_source_by( + primary: Option<&FunctionSignature>, + default_source: Option<&FunctionSignature>, + types_match: impl Fn(&IrType, &IrType) -> bool, + ) -> Option { + let Some(primary) = primary else { + return default_source.cloned(); + }; + let Some(default_source) = default_source else { + return Some(primary.clone()); + }; + let mut merged = primary.clone(); + if Self::params_match_for_default_inheritance(primary, default_source, &types_match) { + for (param, default_param) in merged.params.iter_mut().zip(&default_source.params) { + if param.default.is_none() { + param.default = default_param.default.clone(); + } + } + } + Some(merged) + } + + fn params_match_for_default_inheritance( + left: &FunctionSignature, + right: &FunctionSignature, + types_match: &impl Fn(&IrType, &IrType) -> bool, + ) -> bool { + left.params.len() == right.params.len() + && left + .params + .iter() + .zip(&right.params) + .all(|(left, right)| Self::param_matches_for_default_inheritance(left, right, types_match)) + } + + fn param_matches_for_default_inheritance( + left: &FunctionParam, + right: &FunctionParam, + types_match: &impl Fn(&IrType, &IrType) -> bool, + ) -> bool { + left.kind == right.kind + && types_match(&left.ty, &right.ty) + && (left.name == right.name + || left.name.starts_with("__incan_arg_") + || right.name.starts_with("__incan_arg_")) + } +} + /// Registry of all function signatures in the program #[derive(Debug, Clone, Default)] pub struct FunctionRegistry { @@ -70,22 +149,103 @@ impl FunctionRegistry { Self::default() } + /// Build the registry key used for a canonical module path such as `helpers.normalize`. + pub fn canonical_key(path: &[String]) -> Option { + if path.len() < 2 { + return None; + } + Some(path.join("::")) + } + /// Register a function signature pub fn register(&mut self, name: String, params: Vec, return_type: IrType) { self.signatures.insert(name, FunctionSignature { params, return_type }); } + /// Register a function signature under its canonical module path. + pub fn register_canonical_path(&mut self, path: &[String], params: Vec, return_type: IrType) { + if let Some(key) = Self::canonical_key(path) { + self.register(key, params, return_type); + } + } + /// Look up a function signature by name pub fn get(&self, name: &str) -> Option<&FunctionSignature> { self.signatures.get(name) } + /// Look up a function signature by canonical module path. + pub fn get_canonical_path(&self, path: &[String]) -> Option<&FunctionSignature> { + let key = Self::canonical_key(path)?; + self.signatures.get(&key) + } + + /// Iterate over registered function signatures. + pub fn iter(&self) -> impl Iterator { + self.signatures.iter() + } + /// Merge another registry into this one pub fn merge(&mut self, other: &FunctionRegistry) { for (name, sig) in &other.signatures { self.signatures.insert(name.clone(), sig.clone()); } } + + /// Resolve the effective function-call signature for one IR call site. + /// + /// This is the single merge point for callable metadata during emission. Typechecker/lowering metadata can carry a + /// precise callable surface, while the source registry can carry default expressions. Canonical paths resolve + /// through the cross-module registry, local names resolve through the module registry, and lowered function types + /// are only a final fallback. + pub fn effective_call_signature( + local_registry: &FunctionRegistry, + canonical_registry: &FunctionRegistry, + local_name: Option<&str>, + canonical_path: Option<&[String]>, + callable_signature: Option<&FunctionSignature>, + callee_ty: Option<&IrType>, + ) -> Option { + Self::effective_call_signature_by( + local_registry, + canonical_registry, + local_name, + canonical_path, + callable_signature, + callee_ty, + |left, right| left == right, + ) + } + + /// Resolve the effective function-call signature using a caller-supplied type equivalence rule. + pub fn effective_call_signature_by( + local_registry: &FunctionRegistry, + canonical_registry: &FunctionRegistry, + local_name: Option<&str>, + canonical_path: Option<&[String]>, + callable_signature: Option<&FunctionSignature>, + callee_ty: Option<&IrType>, + types_match: impl Fn(&IrType, &IrType) -> bool, + ) -> Option { + let registry_signature = if let Some(path) = canonical_path { + canonical_registry.get_canonical_path(path) + } else { + local_name.and_then(|name| local_registry.get(name)) + }; + FunctionSignature::merge_default_source_by(callable_signature, registry_signature, types_match).or_else(|| { + match callee_ty { + Some(IrType::Function { params, ret }) => Some(FunctionSignature::from_function_type(params, ret)), + _ => None, + } + }) + } +} + +/// Public source import re-export that should behave like the imported callable for metadata lookups. +#[derive(Debug, Clone)] +pub struct FunctionReexport { + pub name: String, + pub target_path: Vec, } /// A complete IR program @@ -99,6 +259,8 @@ pub struct IrProgram { pub entry_point: Option, /// Function signature registry for call-site type checking pub function_registry: FunctionRegistry, + /// Public source-function re-exports keyed by local exported name and canonical target path. + pub function_reexports: Vec, /// RFC 023: The `rust.module("path::to::module")` Rust backing path, if declared. /// /// When present, `@rust.extern` functions in this program emit delegation calls to this Rust module path instead @@ -119,6 +281,7 @@ impl IrProgram { source_module_name: None, entry_point: None, function_registry: FunctionRegistry::new(), + function_reexports: Vec::new(), rust_module_path: None, newtype_checked_ctor: std::collections::HashMap::new(), } diff --git a/src/backend/ir/ownership.rs b/src/backend/ir/ownership.rs index ee071a198..3a279fba7 100644 --- a/src/backend/ir/ownership.rs +++ b/src/backend/ir/ownership.rs @@ -15,7 +15,7 @@ use super::conversions::{ incan_mutable_param_passed_as_rust_mut_ref, }; use super::decl::FunctionParam; -use super::expr::{IrExpr, IrExprKind, VarAccess}; +use super::expr::{IrExpr, IrExprKind, MethodCallArgPolicy, VarAccess, VarRefKind}; use super::types::IrType; /// A typed sink/source boundary that needs an ownership/coercion decision. @@ -71,6 +71,49 @@ pub enum ValueUseSite<'a> { MethodArg, } +/// Receiver and lookup facts needed to choose the value-use site for one ordinary method-call argument. +/// +/// This keeps clone-bound inference and method emission on the same method-argument boundary decision instead of +/// letting each phase classify receiver ownership independently. +#[derive(Debug, Clone, Copy)] +pub struct RegularMethodArgumentContext { + pub arg_policy: MethodCallArgPolicy, + pub receiver_ref_kind: Option, + pub has_incan_method_signature: bool, + pub is_incan_owned_nominal_receiver: bool, + pub is_rusttype_alias_receiver: bool, + pub preserves_lookup_arg_shape: bool, + pub in_return: bool, +} + +/// Choose the value-use site for an ordinary method-call argument from shared receiver facts. +pub fn regular_method_argument_use_site<'a>( + context: RegularMethodArgumentContext, + callee_param: Option<&'a FunctionParam>, +) -> ValueUseSite<'a> { + let target_ty = callee_param.map(|param| ¶m.ty); + if context.receiver_ref_kind != Some(VarRefKind::ExternalRustName) + && (context.has_incan_method_signature + || (context.is_incan_owned_nominal_receiver && !context.is_rusttype_alias_receiver)) + { + ValueUseSite::IncanCallArg { + target_ty, + callee_param, + in_return: false, + } + } else if context.receiver_ref_kind == Some(VarRefKind::ExternalName) { + ValueUseSite::IncanCallArg { + target_ty, + callee_param, + in_return: context.in_return, + } + } else if matches!(context.arg_policy, MethodCallArgPolicy::PreserveShape) || context.preserves_lookup_arg_shape { + ValueUseSite::MethodArg + } else { + ValueUseSite::ExternalCallArg { target_ty } + } +} + /// Plan how one IR expression should be emitted at a specific ownership boundary. pub fn plan_value_use(expr: &IrExpr, site: ValueUseSite<'_>) -> OwnershipPlan { match site { @@ -110,11 +153,181 @@ pub fn plan_value_use(expr: &IrExpr, site: ValueUseSite<'_>) -> OwnershipPlan { } } +/// Return whether the shared value-use planner requires a backend `.clone()` at this use site. +/// +/// Trait-bound inference uses this as a query-only view of the same ownership decision that expression emission uses +/// before applying a conversion. Keep clone-bound inference going through this API instead of duplicating conversion +/// heuristics in the inference pass. +#[must_use] +pub fn value_use_requires_clone_bound(expr: &IrExpr, site: ValueUseSite<'_>) -> bool { + matches!(plan_value_use(expr, site), OwnershipPlan::Clone) +} + +/// Return the target type carried by a value-use site, if the site has one. +pub fn value_use_site_target_ty<'a>(site: ValueUseSite<'a>) -> Option<&'a IrType> { + match site { + ValueUseSite::IncanCallArg { target_ty, .. } + | ValueUseSite::ExternalCallArg { target_ty } + | ValueUseSite::StructField { target_ty } + | ValueUseSite::CollectionElement { target_ty } + | ValueUseSite::Assignment { target_ty } + | ValueUseSite::ReturnValue { target_ty } + | ValueUseSite::MatchScrutinee { target_ty } => target_ty, + ValueUseSite::MethodArg => None, + } +} + +/// Value-level coercion selected for a callable argument before the final pass-by shape is applied. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ArgumentValuePlan { + /// Apply the ordinary ownership/coercion conversion for this value-use site. + Ownership(OwnershipPlan), + /// Convert `Vec` into `Vec` at an external Rust call boundary. + ExternalListElementInto, +} + +impl ArgumentValuePlan { + /// Apply the value-level plan to an unplanned emitted argument expression. + fn apply_full(&self, tokens: TokenStream) -> TokenStream { + match self { + Self::Ownership(plan) => plan.apply(tokens), + Self::ExternalListElementInto => quote! { + (#tokens).into_iter().map(|__incan_item| ::std::convert::Into::into(__incan_item)).collect::>() + }, + } + } + + /// Apply only value-level work that is not already handled by [`plan_value_use`]. + fn apply_after_value_plan(&self, tokens: TokenStream) -> TokenStream { + match self { + Self::Ownership(_) => tokens, + Self::ExternalListElementInto => self.apply_full(tokens), + } + } +} + +/// Final Rust argument passing shape after value-level ownership/coercion has been handled. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ArgumentPassingMode { + /// Pass the value expression directly. + ByValue, + /// Pass the value expression as `&value`. + SharedBorrow, + /// Pass the value expression as `&mut value`. + MutableBorrow, +} + +impl ArgumentPassingMode { + /// Apply the final argument passing shape. + fn apply(self, tokens: TokenStream) -> TokenStream { + match self { + Self::ByValue => tokens, + Self::SharedBorrow => quote! { &#tokens }, + Self::MutableBorrow => quote! { &mut #tokens }, + } + } +} + +/// Explicit argument-passing plan for a callable argument. +/// +/// Argument emission is intentionally two-stage because some Incan calls need both value-level materialization and a +/// final Rust borrow shape, for example `mut s: str` lowering to `&mut "x".to_string()`. Call emitters should build one +/// of these plans, emit the argument expression, then apply the plan once. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ArgumentPassingPlan { + value: ArgumentValuePlan, + passing: ArgumentPassingMode, +} + +impl ArgumentPassingPlan { + /// Plan one argument at the given use site. + pub fn for_use_site(expr: &IrExpr, site: ValueUseSite<'_>) -> Self { + let mut value = match site { + ValueUseSite::ExternalCallArg { target_ty } + if external_list_arg_needs_element_into(&expr.ty, target_ty) => + { + ArgumentValuePlan::ExternalListElementInto + } + _ => ArgumentValuePlan::Ownership(plan_value_use(expr, site)), + }; + let mut passing = ArgumentPassingMode::ByValue; + + if let IrExprKind::Var { access, .. } = &expr.kind { + match access { + VarAccess::BorrowMut => { + passing = ArgumentPassingMode::MutableBorrow; + value = ArgumentValuePlan::Ownership(OwnershipPlan::None); + } + VarAccess::Borrow if value_use_site_target_ty(site).is_none() => { + passing = ArgumentPassingMode::SharedBorrow; + value = ArgumentValuePlan::Ownership(OwnershipPlan::None); + } + _ => {} + } + } + + if let ValueUseSite::IncanCallArg { + callee_param: Some(param), + .. + } = site + && incan_mutable_param_passed_as_rust_mut_ref(param) + && !matches!(expr.ty, IrType::Ref(_) | IrType::RefMut(_)) + { + passing = ArgumentPassingMode::MutableBorrow; + } + + Self { value, passing } + } + + /// Apply the complete plan to an argument that was emitted without value-use planning. + pub fn apply_full(&self, tokens: TokenStream) -> TokenStream { + self.passing.apply(self.value.apply_full(tokens)) + } + + /// Apply only the portion of the plan that remains after `emit_expr_for_use` or literal seeding already shaped the + /// value. + pub fn apply_after_value_plan(&self, tokens: TokenStream) -> TokenStream { + self.passing.apply(self.value.apply_after_value_plan(tokens)) + } +} + /// Wrapper predicate for mutable aggregate Incan parameters at Rust call sites. pub fn incan_call_arg_needs_rust_mut_borrow(param: &FunctionParam) -> bool { incan_mutable_param_passed_as_rust_mut_ref(param) } +/// Return whether an external Rust list argument needs element-wise `Into` coercion. +fn external_list_arg_needs_element_into(source_ty: &IrType, target_ty: Option<&IrType>) -> bool { + let Some(IrType::List(target_elem)) = target_ty else { + return false; + }; + let IrType::List(source_elem) = source_ty else { + return false; + }; + source_elem != target_elem && !is_unresolved_call_seed_type(target_elem) +} + +/// Return whether a call-seed target still contains unresolved generic or unknown parts. +fn is_unresolved_call_seed_type(ty: &IrType) -> bool { + match ty { + IrType::Unknown | IrType::Generic(_) => true, + IrType::Ref(inner) | IrType::RefMut(inner) | IrType::Option(inner) | IrType::List(inner) => { + is_unresolved_call_seed_type(inner) + } + IrType::Set(inner) => is_unresolved_call_seed_type(inner), + IrType::Dict(key, value) | IrType::Result(key, value) => { + is_unresolved_call_seed_type(key) || is_unresolved_call_seed_type(value) + } + IrType::Tuple(items) => items.iter().any(is_unresolved_call_seed_type), + IrType::NamedGeneric(_, args) => args.iter().any(is_unresolved_call_seed_type), + IrType::Function { params, ret } => { + params.iter().any(is_unresolved_call_seed_type) || is_unresolved_call_seed_type(ret) + } + IrType::Struct(_) | IrType::Enum(_) | IrType::Trait(_) => false, + _ => false, + } +} + /// Whether a collection receiver should be passed through, borrowed, or mutably borrowed. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum CollectionReceiverPlan { @@ -437,6 +650,10 @@ mod tests { use crate::backend::ir::expr::{IrExpr, IrExprKind, VarAccess, VarRefKind}; use crate::backend::ir::types::Mutability; + fn render(tokens: TokenStream) -> String { + tokens.to_string().replace(' ', "") + } + #[test] fn incan_call_string_literal_plans_owned_string() { let expr = IrExpr::new(IrExprKind::String("x".to_string()), IrType::String); @@ -464,6 +681,129 @@ mod tests { assert!(incan_call_arg_needs_rust_mut_borrow(¶m)); } + #[test] + fn argument_plan_mutable_list_param_reborrows_without_value_clone() { + let expr = IrExpr::new( + IrExprKind::Var { + name: "items".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::List(Box::new(IrType::Int)), + ); + let param = FunctionParam { + name: "items".to_string(), + ty: IrType::List(Box::new(IrType::Int)), + mutability: Mutability::Mutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }; + let plan = ArgumentPassingPlan::for_use_site( + &expr, + ValueUseSite::IncanCallArg { + target_ty: Some(¶m.ty), + callee_param: Some(¶m), + in_return: false, + }, + ); + assert_eq!(render(plan.apply_after_value_plan(quote! { items })), "&mutitems"); + } + + #[test] + fn argument_plan_mutable_string_literal_materializes_then_reborrows() { + let expr = IrExpr::new(IrExprKind::String("x".to_string()), IrType::String); + let param = FunctionParam { + name: "s".to_string(), + ty: IrType::String, + mutability: Mutability::Mutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }; + let plan = ArgumentPassingPlan::for_use_site( + &expr, + ValueUseSite::IncanCallArg { + target_ty: Some(¶m.ty), + callee_param: Some(¶m), + in_return: false, + }, + ); + assert_eq!(render(plan.apply_full(quote! { "x" })), "&mut\"x\".to_string()"); + } + + #[test] + fn argument_plan_external_ref_param_borrows_once() { + let expr = IrExpr::new( + IrExprKind::Var { + name: "thing".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("demo::Thing".to_string()), + ); + let target = IrType::Ref(Box::new(IrType::Struct("demo::Thing".to_string()))); + let plan = ArgumentPassingPlan::for_use_site( + &expr, + ValueUseSite::ExternalCallArg { + target_ty: Some(&target), + }, + ); + assert_eq!(render(plan.apply_full(quote! { thing })), "&thing"); + assert_eq!(render(plan.apply_after_value_plan(quote! { &thing })), "&thing"); + } + + #[test] + fn argument_plan_external_list_element_into_is_value_plan() { + let expr = IrExpr::new( + IrExprKind::Var { + name: "items".to_string(), + access: VarAccess::Move, + ref_kind: VarRefKind::Value, + }, + IrType::List(Box::new(IrType::String)), + ); + let target = IrType::List(Box::new(IrType::Struct("demo::Name".to_string()))); + let plan = ArgumentPassingPlan::for_use_site( + &expr, + ValueUseSite::ExternalCallArg { + target_ty: Some(&target), + }, + ); + let rendered = render(plan.apply_full(quote! { items })); + assert!(rendered.contains("items).into_iter().map")); + assert!(rendered.contains("Into::into(__incan_item)")); + } + + #[test] + fn argument_plan_clone_bound_query_follows_shared_incan_arg_policy() { + let receiver = IrExpr::new( + IrExprKind::Var { + name: "other".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("Wrapper".to_string()), + ); + let expr = IrExpr::new( + IrExprKind::Field { + object: Box::new(receiver), + field: "_cursor".to_string(), + }, + IrType::Generic("T".to_string()), + ); + + assert!(value_use_requires_clone_bound( + &expr, + ValueUseSite::IncanCallArg { + target_ty: Some(&IrType::Generic("T".to_string())), + callee_param: None, + in_return: false, + } + )); + assert!(!value_use_requires_clone_bound(&expr, ValueUseSite::MethodArg)); + } + #[test] fn list_shared_receiver_borrows_plain_list() { assert_eq!( diff --git a/src/backend/ir/reference_shape.rs b/src/backend/ir/reference_shape.rs new file mode 100644 index 000000000..596aba446 --- /dev/null +++ b/src/backend/ir/reference_shape.rs @@ -0,0 +1,33 @@ +//! Predicates for IR expressions that already emit Rust reference-shaped values. +//! +//! Ownership and coercion planning may still see these expressions as ordinary Incan surface types. Keep the +//! reference-shape predicate here so conversions, method emission, and argument planning do not drift. + +use super::expr::{IrExpr, IrExprKind}; +use super::types::IrType; + +/// Return whether an IR type is already represented as a Rust reference-like value. +#[must_use] +pub fn type_has_rust_reference_shape(ty: &IrType) -> bool { + matches!( + ty, + IrType::Ref(_) | IrType::RefMut(_) | IrType::StrRef | IrType::StaticStr + ) +} + +/// Return whether an expression already emits a Rust reference-shaped value despite carrying an owned Incan surface +/// type in IR. +#[must_use] +pub fn expr_has_rust_reference_shape(expr: &IrExpr) -> bool { + if type_has_rust_reference_shape(&expr.ty) { + return true; + } + matches!( + &expr.kind, + IrExprKind::MethodCall { + method, + args, + .. + } if args.is_empty() && matches!(method.as_str(), "as_slice" | "as_str") + ) +} diff --git a/src/backend/ir/trait_bound_inference.rs b/src/backend/ir/trait_bound_inference.rs index 17286e2a0..a9238ef80 100644 --- a/src/backend/ir/trait_bound_inference.rs +++ b/src/backend/ir/trait_bound_inference.rs @@ -1,7 +1,7 @@ //! RFC 023: Trait bound inference for generic functions. //! //! This module scans IR function bodies to infer which Rust trait bounds are required on each type parameter based on -//! how the parameter is used (e.g., `==` requires `PartialEq`, f-string interpolation requires `Display`). +//! how the parameter is used (e.g., `==` requires `PartialEq`, display f-string interpolation requires `Display`). //! //! ## Inference rules (from RFC 023) //! @@ -9,7 +9,8 @@ //! | --------------------------- | ------------------------------ | //! | `==`, `!=` | `PartialEq` | //! | `<`, `<=`, `>`, `>=` | `PartialOrd` | -//! | f-string interpolation | `std::fmt::Display` | +//! | f-string `{value}` | `std::fmt::Display` | +//! | f-string `{value:?}` | `std::fmt::Debug` | //! | `+` | `std::ops::Add` | //! | `-` | `std::ops::Sub` | //! | `*` | `std::ops::Mul` | @@ -32,9 +33,12 @@ use super::IrProgram; use super::decl::{FunctionParam, IrDeclKind, IrFunction, IrTraitBound, IrTypeParam}; use super::expr::{ BinOp, FormatPart, IrCallArg, IrDictEntry, IrExpr, IrExprKind, IrGeneratorClause, IrListEntry, MethodCallArgPolicy, - VarAccess, VarRefKind, + VarRefKind, +}; +use super::ownership::{ + RegularMethodArgumentContext, ValueUseSite, regular_method_argument_use_site, value_use_requires_clone_bound, + value_use_site_target_ty, }; -use super::ownership::{ValueUseSite, plan_value_use}; use super::stmt::{IrStmt, IrStmtKind}; use super::types::IrType; @@ -61,6 +65,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { collect_inferred_bounds_for_callable( &func.name, func, + &func.type_params, &trait_decls, &mut function_bounds, &mut function_params, @@ -72,6 +77,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { collect_inferred_bounds_for_callable( &key, method, + &method.type_params, &trait_decls, &mut function_bounds, &mut function_params, @@ -80,6 +86,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { } IrDeclKind::Impl(impl_block) => { for (index, method) in impl_block.methods.iter().enumerate() { + let type_params = callable_inference_type_params(method, Some(&impl_block.type_params)); let key = format!( "impl:{}:{}:{}:{}", impl_block.target_type, @@ -90,6 +97,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { collect_inferred_bounds_for_callable( &key, method, + &type_params, &trait_decls, &mut function_bounds, &mut function_params, @@ -114,6 +122,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { propagate_bounds_for_callable( &func.name, func, + &func.type_params, &snapshot, &function_params, &mut function_bounds, @@ -126,6 +135,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { propagate_bounds_for_callable( &key, method, + &method.type_params, &snapshot, &function_params, &mut function_bounds, @@ -135,6 +145,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { } IrDeclKind::Impl(impl_block) => { for (index, method) in impl_block.methods.iter().enumerate() { + let type_params = callable_inference_type_params(method, Some(&impl_block.type_params)); let key = format!( "impl:{}:{}:{}:{}", impl_block.target_type, @@ -145,6 +156,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { propagate_bounds_for_callable( &key, method, + &type_params, &snapshot, &function_params, &mut function_bounds, @@ -162,38 +174,7 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { } // ---- Pass 3: write inferred bounds back into the IR ---- - for decl in &mut program.declarations { - match &mut decl.kind { - IrDeclKind::Function(func) => { - if let Some(inferred) = function_bounds.remove(&func.name) { - func.type_params = inferred; - } - } - IrDeclKind::Trait(trait_decl) => { - for (index, method) in trait_decl.methods.iter_mut().enumerate() { - let key = format!("trait:{}:{}:{}", trait_decl.name, index, method.name); - if let Some(inferred) = function_bounds.remove(&key) { - method.type_params = inferred; - } - } - } - IrDeclKind::Impl(impl_block) => { - for (index, method) in impl_block.methods.iter_mut().enumerate() { - let key = format!( - "impl:{}:{}:{}:{}", - impl_block.target_type, - impl_block.trait_name.as_deref().unwrap_or(""), - index, - method.name - ); - if let Some(inferred) = function_bounds.remove(&key) { - method.type_params = inferred; - } - } - } - _ => {} - } - } + write_back_callable_bounds(program, &mut function_bounds); // ---- Pass 4: backend-synthesized clone bounds ---- // @@ -211,12 +192,16 @@ pub fn infer_trait_bounds(program: &mut IrProgram) { /// boundaries. fn infer_backend_clone_bounds(program: &mut IrProgram) { let clone_derived_self_params = collect_clone_derived_self_params(program); + let clone_context = BackendCloneInferenceContext::from_program(program); for decl in &mut program.declarations { match &mut decl.kind { - IrDeclKind::Function(func) => { - augment_callable_type_params_for_backend_return_clones(&mut func.type_params, &func.body, None) - } + IrDeclKind::Function(func) => augment_callable_type_params_for_backend_return_clones( + &mut func.type_params, + &func.body, + None, + &clone_context, + ), IrDeclKind::Impl(impl_block) => { let self_clone_params = clone_derived_self_params.get(&impl_block.target_type); for method in &impl_block.methods { @@ -224,6 +209,7 @@ fn infer_backend_clone_bounds(program: &mut IrProgram) { &mut impl_block.type_params, &method.body, self_clone_params, + &clone_context, ); } } @@ -232,6 +218,16 @@ fn infer_backend_clone_bounds(program: &mut IrProgram) { } } +fn callable_inference_type_params(func: &IrFunction, owner_type_params: Option<&[IrTypeParam]>) -> Vec { + let mut type_params = owner_type_params.map_or_else(Vec::new, |params| params.to_vec()); + for type_param in &func.type_params { + if !type_params.iter().any(|existing| existing.name == type_param.name) { + type_params.push(type_param.clone()); + } + } + type_params +} + /// Propagate bounds into one program using already-inferred callable signatures from external programs. /// /// This is used after separate IR programs have already run local bound inference. Imported generic call targets can @@ -283,6 +279,7 @@ fn propagate_trait_bounds_from_signature_maps( propagate_bounds_for_callable( &func.name, func, + &func.type_params, &snapshot, &function_params, &mut function_bounds, @@ -295,6 +292,7 @@ fn propagate_trait_bounds_from_signature_maps( propagate_bounds_for_callable( &key, method, + &method.type_params, &snapshot, &function_params, &mut function_bounds, @@ -304,6 +302,7 @@ fn propagate_trait_bounds_from_signature_maps( } IrDeclKind::Impl(impl_block) => { for (index, method) in impl_block.methods.iter().enumerate() { + let type_params = callable_inference_type_params(method, Some(&impl_block.type_params)); let key = format!( "impl:{}:{}:{}:{}", impl_block.target_type, @@ -314,6 +313,7 @@ fn propagate_trait_bounds_from_signature_maps( propagate_bounds_for_callable( &key, method, + &type_params, &snapshot, &function_params, &mut function_bounds, @@ -500,10 +500,86 @@ fn collect_clone_derived_self_params(program: &IrProgram) -> HashMap, + rusttype_alias_names: HashSet, +} + +#[derive(Clone, Copy)] +struct BackendCallCloneContext<'a> { + callable_signature: Option<&'a super::FunctionSignature>, + in_return: bool, +} + +impl BackendCloneInferenceContext { + fn from_program(program: &IrProgram) -> Self { + let mut incan_nominal_names = HashSet::new(); + let mut rusttype_alias_names = HashSet::new(); + for decl in &program.declarations { + match &decl.kind { + IrDeclKind::Struct(s) => { + incan_nominal_names.insert(s.name.clone()); + } + IrDeclKind::Enum(e) => { + incan_nominal_names.insert(e.name.clone()); + } + IrDeclKind::Trait(trait_decl) => { + incan_nominal_names.insert(trait_decl.name.clone()); + } + IrDeclKind::TypeAlias { + name, + is_rusttype: true, + .. + } => { + incan_nominal_names.insert(name.clone()); + rusttype_alias_names.insert(name.clone()); + } + _ => {} + } + } + Self { + incan_nominal_names, + rusttype_alias_names, + } + } + + fn is_incan_owned_nominal_receiver(&self, receiver_ty: &IrType) -> bool { + match receiver_type_for_method_dispatch(receiver_ty) { + IrType::Struct(name) | IrType::NamedGeneric(name, _) | IrType::Enum(name) => { + self.name_matches(name, &self.incan_nominal_names) + } + IrType::Trait(_) => true, + _ => false, + } + } + + fn is_rusttype_alias_receiver(&self, receiver_ty: &IrType) -> bool { + match receiver_type_for_method_dispatch(receiver_ty) { + IrType::Struct(name) | IrType::NamedGeneric(name, _) => self.name_matches(name, &self.rusttype_alias_names), + _ => false, + } + } + + fn name_matches(&self, name: &str, names: &HashSet) -> bool { + let short_name = name.rsplit("::").next().unwrap_or(name); + names.contains(name) || names.contains(short_name) + } +} + +fn receiver_type_for_method_dispatch(receiver_ty: &IrType) -> &IrType { + let mut receiver_ty = receiver_ty; + while let IrType::Ref(inner) | IrType::RefMut(inner) = receiver_ty { + receiver_ty = inner.as_ref(); + } + receiver_ty +} + fn augment_callable_type_params_for_backend_return_clones( type_params: &mut [IrTypeParam], body: &[IrStmt], self_clone_params: Option<&HashSet>, + clone_context: &BackendCloneInferenceContext, ) { if type_params.is_empty() { return; @@ -512,7 +588,13 @@ fn augment_callable_type_params_for_backend_return_clones( let type_param_names: HashSet<&str> = type_params.iter().map(|tp| tp.name.as_str()).collect(); let mut clone_params = HashSet::new(); for stmt in body { - collect_backend_clone_bounds_in_stmt(stmt, &type_param_names, self_clone_params, &mut clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + &type_param_names, + self_clone_params, + clone_context, + &mut clone_params, + ); } for tp in type_params { @@ -532,6 +614,7 @@ fn collect_backend_clone_bounds_in_stmt( stmt: &IrStmt, type_param_names: &HashSet<&str>, self_clone_params: Option<&HashSet>, + clone_context: &BackendCloneInferenceContext, clone_params: &mut HashSet, ) { match &stmt.kind { @@ -543,6 +626,7 @@ fn collect_backend_clone_bounds_in_stmt( }, type_param_names, self_clone_params, + clone_context, clone_params, ); if let IrExprKind::Call { @@ -555,30 +639,63 @@ fn collect_backend_clone_bounds_in_stmt( collect_backend_clone_bounds_in_call( func, args, - callable_signature.as_ref(), - true, + BackendCallCloneContext { + callable_signature: callable_signature.as_ref(), + in_return: true, + }, type_param_names, self_clone_params, + clone_context, clone_params, ); } else { - collect_backend_clone_bounds_in_expr(expr, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrStmtKind::Expr(expr) => { - collect_backend_clone_bounds_in_expr(expr, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrStmtKind::Let { value, .. } | IrStmtKind::Assign { value, .. } | IrStmtKind::CompoundAssign { value, .. } => { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrStmtKind::While { body, .. } | IrStmtKind::Loop { body, .. } => { for stmt in body { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrStmtKind::For { body, .. } => { for stmt in body { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrStmtKind::If { @@ -587,11 +704,23 @@ fn collect_backend_clone_bounds_in_stmt( .. } => { for stmt in then_branch { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } if let Some(else_branch) = else_branch { for stmt in else_branch { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } @@ -603,23 +732,48 @@ fn collect_backend_clone_bounds_in_stmt( }, type_param_names, self_clone_params, + clone_context, clone_params, ); for arm in arms { if let IrExprKind::Block { stmts, .. } = &arm.body.kind { for stmt in stmts { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } - collect_backend_clone_bounds_in_expr(&arm.body, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arm.body, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(guard) = &arm.guard { - collect_backend_clone_bounds_in_expr(guard, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + guard, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } IrStmtKind::Block(stmts) => { for stmt in stmts { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrStmtKind::Break { value: Some(expr), .. } => { @@ -630,9 +784,16 @@ fn collect_backend_clone_bounds_in_stmt( }, type_param_names, self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + expr, + type_param_names, + self_clone_params, + clone_context, clone_params, ); - collect_backend_clone_bounds_in_expr(expr, type_param_names, self_clone_params, clone_params); } IrStmtKind::Return(None) | IrStmtKind::Break { label: _, value: None } | IrStmtKind::Continue(_) => {} } @@ -648,9 +809,10 @@ fn collect_backend_clone_bounds_for_value_use<'a>( site: ValueUseSite<'a>, type_param_names: &HashSet<&str>, self_clone_params: Option<&HashSet>, + clone_context: &BackendCloneInferenceContext, clone_params: &mut HashSet, ) { - if value_use_requires_backend_clone(expr, site) { + if value_use_requires_clone_bound(expr, site) { add_backend_clone_bounds_for_cloned_expr(expr, type_param_names, self_clone_params, clone_params); } @@ -688,6 +850,7 @@ fn collect_backend_clone_bounds_for_value_use<'a>( item_site, type_param_names, self_clone_params, + clone_context, clone_params, ); } @@ -709,10 +872,17 @@ fn collect_backend_clone_bounds_for_value_use<'a>( }, type_param_names, self_clone_params, + clone_context, clone_params, ), IrListEntry::Spread(value) => { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } @@ -735,6 +905,7 @@ fn collect_backend_clone_bounds_for_value_use<'a>( }, type_param_names, self_clone_params, + clone_context, clone_params, ); collect_backend_clone_bounds_for_value_use( @@ -744,11 +915,18 @@ fn collect_backend_clone_bounds_for_value_use<'a>( }, type_param_names, self_clone_params, + clone_context, clone_params, ); } IrDictEntry::Spread(value) => { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } @@ -760,6 +938,7 @@ fn collect_backend_clone_bounds_for_value_use<'a>( ValueUseSite::StructField { target_ty: None }, type_param_names, self_clone_params, + clone_context, clone_params, ); } @@ -775,38 +954,50 @@ fn collect_backend_clone_bounds_for_value_use<'a>( fn collect_backend_clone_bounds_in_call( func: &IrExpr, args: &[IrCallArg], - callable_signature: Option<&super::FunctionSignature>, - in_return: bool, + call_context: BackendCallCloneContext<'_>, type_param_names: &HashSet<&str>, self_clone_params: Option<&HashSet>, + clone_context: &BackendCloneInferenceContext, clone_params: &mut HashSet, ) { if call_args_use_incan_clone_policy(func) { for (idx, arg) in args.iter().enumerate() { - let sig_param = callable_signature.and_then(|sig| sig.params.get(idx)); + let sig_param = call_context.callable_signature.and_then(|sig| sig.params.get(idx)); let target_ty = sig_param.map(|param| ¶m.ty).or_else(|| match &func.ty { IrType::Function { params, .. } => params.get(idx), _ => None, }); - let requires_clone = value_use_requires_backend_clone( + let requires_clone = value_use_requires_clone_bound( &arg.expr, ValueUseSite::IncanCallArg { target_ty, callee_param: sig_param, - in_return, + in_return: call_context.in_return, }, ); if requires_clone { add_backend_clone_bounds_for_cloned_expr(&arg.expr, type_param_names, self_clone_params, clone_params); } - collect_backend_clone_bounds_in_expr(&arg.expr, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arg.expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } else { for arg in args { - collect_backend_clone_bounds_in_expr(&arg.expr, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arg.expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } - collect_backend_clone_bounds_in_expr(func, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr(func, type_param_names, self_clone_params, clone_context, clone_params); } /// Walk an expression tree for backend-planned clones and explicit clone calls that affect generic bounds. @@ -814,6 +1005,7 @@ fn collect_backend_clone_bounds_in_expr( expr: &IrExpr, type_param_names: &HashSet<&str>, self_clone_params: Option<&HashSet>, + clone_context: &BackendCloneInferenceContext, clone_params: &mut HashSet, ) { match &expr.kind { @@ -821,31 +1013,64 @@ fn collect_backend_clone_bounds_in_expr( receiver, args, arg_policy, + callable_signature, .. } => { - if method_call_args_use_incan_clone_policy(receiver, *arg_policy) { - for arg in args { - if incan_call_arg_requires_backend_clone(&arg.expr) { - add_backend_clone_bounds_for_cloned_expr( - &arg.expr, - type_param_names, - self_clone_params, - clone_params, - ); - } - collect_backend_clone_bounds_in_expr(&arg.expr, type_param_names, self_clone_params, clone_params); - } - } else { - for arg in args { - collect_backend_clone_bounds_in_expr(&arg.expr, type_param_names, self_clone_params, clone_params); + let callable_signature = callable_signature.as_ref(); + for (idx, arg) in args.iter().enumerate() { + let sig_param = callable_signature.and_then(|sig| sig.params.get(idx)); + let use_site = regular_method_argument_use_site( + RegularMethodArgumentContext { + arg_policy: *arg_policy, + receiver_ref_kind: receiver_ref_kind(receiver), + has_incan_method_signature: callable_signature.is_some(), + is_incan_owned_nominal_receiver: clone_context.is_incan_owned_nominal_receiver(&receiver.ty), + is_rusttype_alias_receiver: clone_context.is_rusttype_alias_receiver(&receiver.ty), + preserves_lookup_arg_shape: matches!(arg_policy, MethodCallArgPolicy::PreserveShape), + in_return: false, + }, + sig_param, + ); + if value_use_requires_clone_bound(&arg.expr, use_site) { + add_backend_clone_bounds_for_cloned_expr( + &arg.expr, + type_param_names, + self_clone_params, + clone_params, + ); } + collect_backend_clone_bounds_in_expr( + &arg.expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } - collect_backend_clone_bounds_in_expr(receiver, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + receiver, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrExprKind::KnownMethodCall { receiver, args, .. } => { - collect_backend_clone_bounds_in_expr(receiver, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + receiver, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); for arg in args { - collect_backend_clone_bounds_in_expr(&arg.expr, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arg.expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Call { @@ -856,22 +1081,37 @@ fn collect_backend_clone_bounds_in_expr( } => collect_backend_clone_bounds_in_call( func, args, - callable_signature.as_ref(), - false, + BackendCallCloneContext { + callable_signature: callable_signature.as_ref(), + in_return: false, + }, type_param_names, self_clone_params, + clone_context, clone_params, ), IrExprKind::BuiltinCall { args, .. } | IrExprKind::Tuple(args) => { for arg in args { - collect_backend_clone_bounds_in_expr(arg, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + arg, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::List(args) => { for arg in args { match arg { IrListEntry::Element(value) | IrListEntry::Spread(value) => { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } @@ -880,23 +1120,53 @@ fn collect_backend_clone_bounds_in_expr( for entry in entries { match entry { IrDictEntry::Pair(key, value) => { - collect_backend_clone_bounds_in_expr(key, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + key, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrDictEntry::Spread(value) => { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } } IrExprKind::Set(items) => { for item in items { - collect_backend_clone_bounds_in_expr(item, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + item, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Struct { fields, .. } => { for (_, value) in fields { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Field { object, .. } @@ -906,15 +1176,33 @@ fn collect_backend_clone_bounds_in_expr( | IrExprKind::NumericResize { expr: object, .. } | IrExprKind::InteropCoerce { expr: object, .. } | IrExprKind::UnaryOp { operand: object, .. } => { - collect_backend_clone_bounds_in_expr(object, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + object, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrExprKind::BinOp { left, right, .. } | IrExprKind::Index { object: left, index: right, } => { - collect_backend_clone_bounds_in_expr(left, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(right, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + left, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + right, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrExprKind::Slice { target, @@ -922,15 +1210,39 @@ fn collect_backend_clone_bounds_in_expr( end, step, } => { - collect_backend_clone_bounds_in_expr(target, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + target, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(start) = start { - collect_backend_clone_bounds_in_expr(start, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + start, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } if let Some(end) = end { - collect_backend_clone_bounds_in_expr(end, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + end, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } if let Some(step) = step { - collect_backend_clone_bounds_in_expr(step, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + step, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::If { @@ -938,29 +1250,77 @@ fn collect_backend_clone_bounds_in_expr( then_branch, else_branch, } => { - collect_backend_clone_bounds_in_expr(condition, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(then_branch, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + condition, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + then_branch, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(else_branch) = else_branch { - collect_backend_clone_bounds_in_expr(else_branch, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + else_branch, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Block { stmts, value } => { for stmt in stmts { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } if let Some(value) = value { - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Loop { body } => { for stmt in body { - collect_backend_clone_bounds_in_stmt(stmt, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_stmt( + stmt, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Race { arms, .. } => { for arm in arms { - collect_backend_clone_bounds_in_expr(&arm.awaitable, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(&arm.body, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arm.awaitable, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + &arm.body, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Match { scrutinee, arms } => { @@ -971,17 +1331,36 @@ fn collect_backend_clone_bounds_in_expr( }, type_param_names, self_clone_params, + clone_context, clone_params, ); for arm in arms { - collect_backend_clone_bounds_in_expr(&arm.body, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + &arm.body, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(guard) = &arm.guard { - collect_backend_clone_bounds_in_expr(guard, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + guard, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } IrExprKind::Closure { body, .. } => { - collect_backend_clone_bounds_in_expr(body, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + body, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } IrExprKind::ListComp { element, @@ -989,10 +1368,28 @@ fn collect_backend_clone_bounds_in_expr( filter, .. } => { - collect_backend_clone_bounds_in_expr(element, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(iterable, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + element, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + iterable, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(filter) = filter { - collect_backend_clone_bounds_in_expr(filter, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + filter, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::DictComp { @@ -1002,15 +1399,39 @@ fn collect_backend_clone_bounds_in_expr( filter, .. } => { - collect_backend_clone_bounds_in_expr(key, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(value, type_param_names, self_clone_params, clone_params); - collect_backend_clone_bounds_in_expr(iterable, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr(key, type_param_names, self_clone_params, clone_context, clone_params); + collect_backend_clone_bounds_in_expr( + value, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); + collect_backend_clone_bounds_in_expr( + iterable, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); if let Some(filter) = filter { - collect_backend_clone_bounds_in_expr(filter, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + filter, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Generator { element, clauses } => { - collect_backend_clone_bounds_in_expr(element, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + element, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); for clause in clauses { match clause { IrGeneratorClause::For { iterable, .. } => { @@ -1018,6 +1439,7 @@ fn collect_backend_clone_bounds_in_expr( iterable, type_param_names, self_clone_params, + clone_context, clone_params, ); } @@ -1026,6 +1448,7 @@ fn collect_backend_clone_bounds_in_expr( condition, type_param_names, self_clone_params, + clone_context, clone_params, ); } @@ -1034,16 +1457,34 @@ fn collect_backend_clone_bounds_in_expr( } IrExprKind::Range { start, end, .. } => { if let Some(start) = start { - collect_backend_clone_bounds_in_expr(start, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + start, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } if let Some(end) = end { - collect_backend_clone_bounds_in_expr(end, type_param_names, self_clone_params, clone_params); + collect_backend_clone_bounds_in_expr( + end, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } IrExprKind::Format { parts } => { for part in parts { - if let FormatPart::Expr(expr) = part { - collect_backend_clone_bounds_in_expr(expr, type_param_names, self_clone_params, clone_params); + if let FormatPart::Expr { expr, .. } = part { + collect_backend_clone_bounds_in_expr( + expr, + type_param_names, + self_clone_params, + clone_context, + clone_params, + ); } } } @@ -1067,24 +1508,11 @@ fn collect_backend_clone_bounds_in_expr( } } -/// Return whether the shared ownership planner would emit `.clone()` for this exact use site. -fn value_use_requires_backend_clone(expr: &IrExpr, site: ValueUseSite<'_>) -> bool { - matches!(plan_value_use(expr, site), super::conversions::Conversion::Clone) -} - -/// Return whether method-call arguments should follow owned Incan call semantics. -/// -/// External Rust receivers and preserve-shape methods own their argument borrowing rules, so applying the generic Incan -/// clone heuristic there would over-constrain generated signatures. -fn method_call_args_use_incan_clone_policy(receiver: &IrExpr, arg_policy: MethodCallArgPolicy) -> bool { - !matches!(arg_policy, MethodCallArgPolicy::PreserveShape) - && !matches!( - &receiver.kind, - IrExprKind::Var { - ref_kind: VarRefKind::ExternalRustName, - .. - } - ) +fn receiver_ref_kind(receiver: &IrExpr) -> Option { + match &receiver.kind { + IrExprKind::Var { ref_kind, .. } => Some(*ref_kind), + _ => None, + } } /// Return whether a call expression targets an Incan callable rather than an external Rust symbol. @@ -1112,34 +1540,6 @@ fn borrowed_method_inner_ty(expr: &IrExpr) -> Option<&IrType> { } } -/// Lightweight predicate for Incan call arguments that may clone before the full use-site planner runs. -/// -/// This covers the common clone-producing shapes used for call arguments: non-last-use non-`Copy` variables, -/// non-`Copy` field reads, borrowed non-`Copy` values, and `as_ref()` results that expose non-`Copy` inner data. -fn incan_call_arg_requires_backend_clone(expr: &IrExpr) -> bool { - match &expr.kind { - IrExprKind::Var { access, .. } if !expr.ty.is_copy() => !matches!(access, VarAccess::Move), - IrExprKind::Field { .. } if !expr.ty.is_copy() => true, - _ if matches!(&expr.ty, IrType::Ref(inner) | IrType::RefMut(inner) if !inner.as_ref().is_copy()) => true, - _ if borrowed_method_inner_ty(expr).is_some_and(|inner| !inner.is_copy()) => true, - _ => false, - } -} - -/// Return the target type carried by a use site, if that site has one. -fn value_use_site_target_ty(site: ValueUseSite<'_>) -> Option<&IrType> { - match site { - ValueUseSite::IncanCallArg { target_ty, .. } - | ValueUseSite::ExternalCallArg { target_ty } - | ValueUseSite::StructField { target_ty } - | ValueUseSite::CollectionElement { target_ty } - | ValueUseSite::Assignment { target_ty } - | ValueUseSite::ReturnValue { target_ty } - | ValueUseSite::MatchScrutinee { target_ty } => target_ty, - ValueUseSite::MethodArg => None, - } -} - /// Rebuild a parent value-use site for one tuple item while preserving the parent ownership context. /// /// Tuple elements can be planned as call arguments, return values, collection elements, and match scrutinees. This @@ -1270,19 +1670,20 @@ fn collect_generic_type_param_names(ty: &IrType, type_param_names: &HashSet<&str fn collect_inferred_bounds_for_callable( key: &str, func: &IrFunction, + type_params: &[IrTypeParam], trait_decls: &HashMap, function_bounds: &mut HashMap>, function_params: &mut HashMap>, ) { - if func.type_params.is_empty() { + if type_params.is_empty() { return; } - let mut inferred = infer_function_bounds(func); + let mut inferred = infer_function_bounds(func, type_params); // Also check return types like `-> DataSet[T]` / `-> BoundedDataSet[T]`, which lower to `impl Trait` and // must carry through any bounds required by the returned trait's generic arguments. - add_bounds_from_return_type(&func.return_type, &func.type_params, trait_decls, &mut inferred); + add_bounds_from_return_type(&func.return_type, type_params, trait_decls, &mut inferred); function_bounds.insert(key.to_string(), inferred); function_params.insert(key.to_string(), func.params.clone()); @@ -1295,16 +1696,17 @@ fn collect_inferred_bounds_for_callable( fn propagate_bounds_for_callable( key: &str, func: &IrFunction, + type_params: &[IrTypeParam], snapshot: &HashMap>, function_params: &HashMap>, function_bounds: &mut HashMap>, changed: &mut bool, ) { - if func.type_params.is_empty() { + if type_params.is_empty() { return; } - let called_generics = collect_called_generic_functions(func, snapshot, function_params); + let called_generics = collect_called_generic_functions(func, type_params, snapshot, function_params); if let Some(current_bounds) = function_bounds.get_mut(key) { for (callee_name, type_arg_mapping) in &called_generics { if let Some(callee_bounds) = snapshot.get(callee_name) @@ -1317,12 +1719,12 @@ fn propagate_bounds_for_callable( } /// Infer trait bounds for a single function by scanning its body. -fn infer_function_bounds(func: &IrFunction) -> Vec { - let type_param_names: HashSet<&str> = func.type_params.iter().map(|tp| tp.name.as_str()).collect(); +fn infer_function_bounds(func: &IrFunction, type_params: &[IrTypeParam]) -> Vec { + let type_param_names: HashSet<&str> = type_params.iter().map(|tp| tp.name.as_str()).collect(); let mut bounds_map: HashMap> = HashMap::new(); // Start with explicit bounds from `with` clauses. - for tp in &func.type_params { + for tp in type_params { bounds_map.insert(tp.name.clone(), tp.bounds.clone()); } @@ -1332,7 +1734,7 @@ fn infer_function_bounds(func: &IrFunction) -> Vec { } // Rebuild type params with combined bounds. - func.type_params + type_params .iter() .map(|tp| { let bounds = bounds_map.remove(&tp.name).unwrap_or_default(); @@ -1437,12 +1839,24 @@ fn scan_expr_for_bounds( scan_expr_for_bounds(right, type_params, params, bounds_map); } - // ---- f-string interpolation: expressions used in format require Display ---- + // ---- f-string interpolation: expressions used in format require the matching formatting trait ---- IrExprKind::Format { parts } => { for part in parts { - if let FormatPart::Expr(inner) = part { + if let FormatPart::Expr { expr: inner, style } = part { + let bound = if style.emits_rust_debug(&inner.ty) { + tb::DEBUG + } else { + tb::DISPLAY + }; + let mut formatted_type_params = HashSet::new(); if let Some(tp_name) = expr_type_param_name(inner, type_params, params) { - add_bound(bounds_map, &tp_name, IrTraitBound::simple(tb::DISPLAY)); + formatted_type_params.insert(tp_name); + } + if style.emits_rust_debug(&inner.ty) { + collect_generic_type_param_names(&inner.ty, type_params, &mut formatted_type_params); + } + for tp_name in formatted_type_params { + add_bound(bounds_map, &tp_name, IrTraitBound::simple(bound)); } scan_expr_for_bounds(inner, type_params, params, bounds_map); } @@ -1764,6 +2178,7 @@ fn expr_type_param_name( fn type_param_name_from_ir_type(ty: &IrType, type_params: &HashSet<&str>) -> Option { match ty { IrType::Generic(name) if type_params.contains(name.as_str()) => Some(name.clone()), + IrType::Struct(name) if type_params.contains(name.as_str()) => Some(name.clone()), _ => None, } } @@ -2074,10 +2489,11 @@ fn substitute_ir_type(ty: &IrType, subst: &HashMap<&str, &IrType>) -> IrType { /// the caller's type parameter names when the argument is a direct type parameter pass-through. fn collect_called_generic_functions( func: &IrFunction, + type_params: &[IrTypeParam], function_bounds: &HashMap>, function_params: &HashMap>, ) -> Vec<(String, HashMap)> { - let type_param_names: HashSet<&str> = func.type_params.iter().map(|tp| tp.name.as_str()).collect(); + let type_param_names: HashSet<&str> = type_params.iter().map(|tp| tp.name.as_str()).collect(); let mut result = Vec::new(); for stmt in &func.body { @@ -2259,8 +2675,8 @@ fn collect_calls_in_expr( } IrExprKind::Format { parts } => { for part in parts { - if let FormatPart::Expr(e) = part { - recurse_expr(e, result); + if let FormatPart::Expr { expr, .. } = part { + recurse_expr(expr, result); } } } @@ -2288,6 +2704,9 @@ fn collect_calls_in_expr( recurse_stmt(stmt, result); } } + IrExprKind::Closure { body, .. } => { + recurse_expr(body, result); + } IrExprKind::Race { arms, .. } => { for arm in arms { recurse_expr(&arm.awaitable, result); @@ -2334,8 +2753,9 @@ fn propagate_transitive_bounds( #[cfg(test)] mod tests { use super::*; - use crate::backend::ir::FunctionRegistry; - use crate::backend::ir::decl::{IrDecl, IrDeclKind, Visibility}; + use crate::backend::ir::decl::{FunctionParam, IrDecl, IrDeclKind, IrImpl, Visibility}; + use crate::backend::ir::expr::{FormatStyle, IrCallArgKind, MethodCallArgPolicy, VarAccess}; + use crate::backend::ir::{FunctionRegistry, FunctionSignature, Mutability, TypedExpr}; fn function(name: &str, type_params: Vec) -> IrFunction { IrFunction { @@ -2362,11 +2782,207 @@ mod tests { source_module_name: None, entry_point: None, function_registry: FunctionRegistry::new(), + function_reexports: Vec::new(), rust_module_path: None, newtype_checked_ctor: Default::default(), } } + #[test] + fn impl_owner_generic_bounds_are_written_to_impl_header() -> Result<(), Box> { + let method = IrFunction { + name: "render".to_string(), + params: Vec::new(), + return_type: IrType::Unit, + body: vec![IrStmt::new(IrStmtKind::Expr(TypedExpr::new( + IrExprKind::Format { + parts: vec![FormatPart::Expr { + expr: TypedExpr::new( + IrExprKind::Var { + name: "value".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Generic("T".to_string()), + ), + style: FormatStyle::Display, + }], + }, + IrType::String, + )))], + is_async: false, + is_generator: false, + visibility: Visibility::Public, + type_params: Vec::new(), + is_extern: false, + rust_attributes: Vec::new(), + lint_allows: Vec::new(), + }; + let mut program = IrProgram { + declarations: vec![IrDecl::new(IrDeclKind::Impl(IrImpl { + target_type: "Boxed".to_string(), + type_params: vec![IrTypeParam::bare("T")], + trait_name: None, + trait_type_args: Vec::new(), + associated_types: Vec::new(), + methods: vec![method], + }))], + source_module_name: None, + entry_point: None, + function_registry: FunctionRegistry::new(), + function_reexports: Vec::new(), + rust_module_path: None, + newtype_checked_ctor: Default::default(), + }; + + infer_trait_bounds(&mut program); + + let decl = program + .declarations + .first() + .ok_or_else(|| std::io::Error::other("expected impl declaration"))?; + let IrDecl { + kind: IrDeclKind::Impl(impl_block), + .. + } = decl + else { + return Err(std::io::Error::other("expected impl declaration").into()); + }; + let bounds = &impl_block.type_params[0].bounds; + assert!( + bounds.contains(&IrTraitBound::simple(tb::DISPLAY)), + "owner generic T should receive Display bound from impl method body, got {bounds:?}" + ); + assert!( + impl_block.methods[0].type_params.is_empty(), + "impl-owner generics must stay on the impl header, not the method signature" + ); + Ok(()) + } + + #[test] + fn backend_clone_bounds_do_not_use_incan_policy_for_external_nominal_methods() + -> Result<(), Box> { + let mut func = function("send", vec![IrTypeParam::bare("T")]); + func.body = vec![IrStmt::new(IrStmtKind::Expr(TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "client".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("external_crate::Client".to_string()), + )), + method: "send".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "value".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Generic("T".to_string()), + ), + }], + callable_signature: None, + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Unit, + )))]; + let mut program = program(vec![func]); + + infer_trait_bounds(&mut program); + + let decl = program + .declarations + .first() + .ok_or_else(|| std::io::Error::other("expected function declaration"))?; + let IrDecl { + kind: IrDeclKind::Function(func), + .. + } = decl + else { + return Err(std::io::Error::other("expected function declaration").into()); + }; + assert!( + func.type_params[0].bounds.is_empty(), + "external nominal method args should not inherit Incan clone policy, got {:?}", + func.type_params[0].bounds + ); + Ok(()) + } + + #[test] + fn backend_clone_bounds_use_incan_policy_for_methods_with_signatures() -> Result<(), Box> { + let mut func = function("send", vec![IrTypeParam::bare("T")]); + func.body = vec![IrStmt::new(IrStmtKind::Expr(TypedExpr::new( + IrExprKind::MethodCall { + receiver: Box::new(TypedExpr::new( + IrExprKind::Var { + name: "client".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Struct("Client".to_string()), + )), + method: "send".to_string(), + dispatch: None, + type_args: Vec::new(), + args: vec![IrCallArg { + name: None, + kind: IrCallArgKind::Positional, + expr: TypedExpr::new( + IrExprKind::Var { + name: "value".to_string(), + access: VarAccess::Read, + ref_kind: VarRefKind::Value, + }, + IrType::Generic("T".to_string()), + ), + }], + callable_signature: Some(FunctionSignature { + params: vec![FunctionParam { + name: "value".to_string(), + ty: IrType::Generic("T".to_string()), + mutability: Mutability::Immutable, + is_self: false, + kind: crate::frontend::ast::ParamKind::Normal, + default: None, + }], + return_type: IrType::Unit, + }), + arg_policy: MethodCallArgPolicy::Default, + }, + IrType::Unit, + )))]; + let mut program = program(vec![func]); + + infer_trait_bounds(&mut program); + + let decl = program + .declarations + .first() + .ok_or_else(|| std::io::Error::other("expected function declaration"))?; + let IrDecl { + kind: IrDeclKind::Function(func), + .. + } = decl + else { + return Err(std::io::Error::other("expected function declaration").into()); + }; + assert!( + func.type_params[0].bounds.contains(&IrTraitBound::simple(tb::CLONE)), + "Incan method signatures should keep clone-bound inference aligned with emission, got {:?}", + func.type_params[0].bounds + ); + Ok(()) + } + #[test] fn external_generic_bounds_do_not_rewrite_same_named_local_non_generic_function() -> Result<(), Box> { diff --git a/src/backend/project/cargo_toml.rs b/src/backend/project/cargo_toml.rs index 5457ecac0..0efb08401 100644 --- a/src/backend/project/cargo_toml.rs +++ b/src/backend/project/cargo_toml.rs @@ -251,10 +251,11 @@ impl ProjectGenerator { }; // ---- Build bin/lib target ---- + let target_name = self.cargo_target_name(); let (bin, lib) = if self.is_binary { ( vec![BinTarget { - name: self.name.clone(), + name: target_name, path: "src/main.rs".into(), }], None, @@ -263,7 +264,7 @@ impl ProjectGenerator { ( vec![], Some(LibTarget { - name: self.name.clone(), + name: target_name, path: "src/lib.rs".into(), }), ) diff --git a/src/backend/project/generator.rs b/src/backend/project/generator.rs index 7df69e71c..521a30766 100644 --- a/src/backend/project/generator.rs +++ b/src/backend/project/generator.rs @@ -14,8 +14,10 @@ use std::path::{Path, PathBuf}; use crate::manifest::DependencySpec; use incan_core::lang::rust_keywords; +use sha2::{Digest as _, Sha256}; const MOD_INSERT_MARKER: &str = "// __INCAN_INSERT_MODS__"; +pub(crate) const GENERATED_CARGO_TARGET_DIR_ENV: &str = "INCAN_GENERATED_CARGO_TARGET_DIR"; // ============================================================================ // RFC 023: Stdlib module naming @@ -151,6 +153,79 @@ impl ProjectGenerator { self.run_profile = profile; } + /// Resolve the optional generated-project Cargo target override. + /// + /// This is primarily used by integration tests and smoke gates that compile many generated Rust projects from one + /// parent workspace. It lets those projects share dependency artifacts while keeping ordinary user invocations on + /// the parent-scoped default target directory. + pub(super) fn generated_cargo_target_dir_override() -> Option { + let raw = std::env::var_os(GENERATED_CARGO_TARGET_DIR_ENV)?; + let raw = PathBuf::from(raw); + if raw.as_os_str().is_empty() { + return None; + } + Some(Self::resolve_target_dir(raw)) + } + + pub(super) fn resolve_target_dir(target_dir: PathBuf) -> PathBuf { + if target_dir.is_absolute() { + target_dir + } else if let Ok(cwd) = std::env::current_dir() { + cwd.join(target_dir) + } else { + target_dir + } + } + + /// Cargo target name used for the generated binary or library target. + /// + /// When a caller opts into a broad shared target directory, multiple unrelated generated projects can have the same + /// user-facing project name (`main`, `consumer`, etc.). Cargo writes root binaries and libraries at + /// `target//`, so shared target dirs need a unique target name to avoid stale binary reuse + /// and parallel build collisions. Library target names stay stable because native Rust consumers import them as + /// crate names from generated library artifacts. + pub(super) fn cargo_target_name(&self) -> String { + if self.is_binary && Self::generated_cargo_target_dir_override().is_some() { + Self::shared_target_safe_name(&self.name, &self.output_dir) + } else { + self.name.clone() + } + } + + pub(super) fn shared_target_safe_name(name: &str, output_dir: &Path) -> String { + let mut normalized = name + .chars() + .map(|ch| if ch.is_ascii_alphanumeric() { ch } else { '_' }) + .collect::(); + if normalized.is_empty() { + normalized.push_str("incan_project"); + } + if !normalized + .as_bytes() + .first() + .is_some_and(|byte| byte.is_ascii_alphabetic() || *byte == b'_') + { + normalized.insert(0, '_'); + } + + let absolute_output_dir = if output_dir.is_absolute() { + output_dir.to_path_buf() + } else if let Ok(cwd) = std::env::current_dir() { + cwd.join(output_dir) + } else { + output_dir.to_path_buf() + }; + + let mut hasher = Sha256::new(); + hasher.update(name.as_bytes()); + hasher.update(b"\0"); + hasher.update(absolute_output_dir.to_string_lossy().as_bytes()); + let digest_bytes = hasher.finalize(); + let digest = hex::encode(&digest_bytes[..8]); + + format!("{normalized}_{digest}") + } + /// Ensure the generated `src/` directory exists. fn ensure_generated_src_dir(&self) -> io::Result { let src_dir = self.output_dir.join("src"); diff --git a/src/backend/project/runner.rs b/src/backend/project/runner.rs index fafa0dbee..a041eda3a 100644 --- a/src/backend/project/runner.rs +++ b/src/backend/project/runner.rs @@ -46,16 +46,14 @@ impl ProjectGenerator { /// tests, and benchmark checks. Sharing a parent-scoped target dir lets those generated crates reuse compiled /// dependencies. fn cargo_target_dir(&self) -> PathBuf { + if let Some(target_dir) = Self::generated_cargo_target_dir_override() { + return target_dir; + } + let base_dir = self.output_dir.parent().unwrap_or(self.output_dir.as_path()); let target_dir = base_dir.join(".cargo-target"); - if target_dir.is_absolute() { - target_dir - } else if let Ok(cwd) = std::env::current_dir() { - cwd.join(target_dir) - } else { - target_dir - } + Self::resolve_target_dir(target_dir) } /// Build the project using cargo. @@ -167,14 +165,14 @@ impl ProjectGenerator { /// Get the path to the built binary. pub fn binary_path(&self) -> PathBuf { - self.cargo_target_dir().join("release").join(&self.name) + self.cargo_target_dir().join("release").join(self.cargo_target_name()) } /// Get the path to the binary produced for `incan run`. pub fn run_binary_path(&self) -> PathBuf { self.cargo_target_dir() .join(self.run_profile_binary_dir()) - .join(&self.name) + .join(self.cargo_target_name()) } } @@ -258,4 +256,28 @@ mod tests { ); Ok(()) } + + #[test] + fn shared_target_safe_name_distinguishes_same_project_name_by_output_dir() -> Result<(), Box> + { + let tmp = tempfile::tempdir()?; + let first = ProjectGenerator::shared_target_safe_name("demo-app", &tmp.path().join("one")); + let second = ProjectGenerator::shared_target_safe_name("demo-app", &tmp.path().join("two")); + + assert_ne!(first, second); + assert!(first.starts_with("demo_app_"), "unexpected target name: {first}"); + assert!( + first.chars().all(|ch| ch.is_ascii_alphanumeric() || ch == '_'), + "target name should be Rust-identifier safe: {first}" + ); + Ok(()) + } + + #[test] + fn relative_target_dirs_resolve_against_current_working_dir() -> Result<(), Box> { + let cwd = std::env::current_dir()?; + let target_dir = ProjectGenerator::resolve_target_dir(PathBuf::from("target/shared-generated")); + assert_eq!(target_dir, cwd.join("target/shared-generated")); + Ok(()) + } } diff --git a/src/cli/commands/build.rs b/src/cli/commands/build.rs index 9495b51e0..a91cdcc34 100644 --- a/src/cli/commands/build.rs +++ b/src/cli/commands/build.rs @@ -10,10 +10,10 @@ use std::path::{Path, PathBuf}; use crate::backend::{IrCodegen, ProjectGenerator, RunProfile}; use crate::cli::{CliError, CliResult, ExitCode}; -use crate::dependency_resolver::resolve_dependencies; +use crate::dependency_resolver::{resolve_dependencies, resolve_reachable_dependencies}; use crate::frontend::api_metadata::{ CHECKED_API_METADATA_SCHEMA_VERSION, CheckedApiMetadataPackage, CheckedApiPackageIdentity, - collect_checked_api_metadata, validate_checked_api_docstrings, + collect_checked_api_metadata, materialize_api_alias_projections, validate_checked_api_docstrings, }; use crate::frontend::ast::{Declaration, Decorator, ImportKind, Span, Spanned}; use crate::frontend::contract_metadata::{ContractMetadataPackage, read_project_model_bundles}; @@ -28,8 +28,8 @@ use crate::lockfile::CargoFeatureSelection; use crate::manifest::ProjectManifest; use super::common::{ - CargoPolicy, build_source_map, cargo_command_flags, collect_inline_rust_imports, collect_modules, - collect_project_requirements, enforce_project_toolchain_constraint, format_dependency_error, + CargoPolicy, build_source_map, cargo_command_flags, collect_modules, collect_project_requirements, + collect_rust_dependency_uses, enforce_project_toolchain_constraint, format_dependency_error, imported_module_deps_for_with_index, merge_project_requirement_dependencies, module_key_index, resolve_project_root, typecheck_modules_with_import_graph, validate_output_dir, }; @@ -544,9 +544,9 @@ fn prepare_project( .and_then(|m| m.build.as_ref().and_then(|b| b.rust_edition.clone())), ); - let mut inline_imports = collect_inline_rust_imports(main_module, false); + let mut inline_imports = collect_rust_dependency_uses(main_module, false); for module in dep_modules { - inline_imports.extend(collect_inline_rust_imports(module, false)); + inline_imports.extend(collect_rust_dependency_uses(module, false)); } // RFC 023: Stdlib modules should not have inline rust imports (they use rust.module() + @rust.extern instead), // so we skip collecting from them. @@ -558,7 +558,7 @@ fn prepare_project( } .normalized(); - let mut resolved = match resolve_dependencies(manifest.as_ref(), &inline_imports, true, &cargo_features) { + let mut resolved = match resolve_reachable_dependencies(manifest.as_ref(), &inline_imports, true, &cargo_features) { Ok(resolved) => resolved, Err(errors) => { let mut msg = String::new(); @@ -720,9 +720,9 @@ pub fn build_library( let rust_extern_contexts = collect_rust_extern_contexts(&modules); let dep_modules = &modules[..modules.len() - 1]; - let mut inline_imports = collect_inline_rust_imports(lib_module, false); + let mut inline_imports = collect_rust_dependency_uses(lib_module, false); for module in dep_modules { - inline_imports.extend(collect_inline_rust_imports(module, false)); + inline_imports.extend(collect_rust_dependency_uses(module, false)); } let project_name = manifest .project @@ -771,10 +771,8 @@ pub fn build_library( rust_inspect_query_paths: &metadata_query_paths, })?; #[cfg(feature = "rust_inspect")] - let rust_inspect_manifest_dir = project_root.join("target").join("incan_lock"); - #[cfg(feature = "rust_inspect")] - { - ensure_rust_inspect_workspace( + let rust_inspect_manifest_dir = { + let rust_inspect_manifest_dir = ensure_rust_inspect_workspace( &project_root, project_name.as_str(), manifest.build.as_ref().and_then(|build| build.rust_edition.clone()), @@ -783,7 +781,8 @@ pub fn build_library( lock_payload_for_typecheck.clone(), )?; prewarm_rust_inspect_workspace(&rust_inspect_manifest_dir, &metadata_query_paths)?; - } + rust_inspect_manifest_dir + }; let mut all_errors = String::new(); let mut checked_exports_by_module: HashMap> = HashMap::new(); @@ -836,6 +835,8 @@ pub fn build_library( return Err(CliError::failure(all_errors.trim_end())); } + materialize_api_alias_projections(&mut api_metadata_modules); + for diagnostic in validate_checked_api_docstrings(&api_metadata_modules) { if let Some(module) = modules .iter() @@ -1116,6 +1117,52 @@ mod tests { assert!(rendered.contains("incan_stdlib::testing::fail")); } + #[test] + fn run_entrypoint_omits_unused_manifest_rust_dependencies() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let project_root = tmp.path(); + let scripts_dir = project_root.join("scripts"); + std::fs::create_dir_all(&scripts_dir)?; + std::fs::write( + project_root.join("incan.toml"), + "[project]\nname = \"unused_rust_dep_run_repro\"\nversion = \"0.1.0\"\n\n[rust-dependencies]\ndatafusion = \"53\"\n", + )?; + std::fs::write( + scripts_dir.join("check.incn"), + "def main() -> None:\n println(\"ok\")\n", + )?; + + let cargo_lock_payload = std::fs::read_to_string(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("Cargo.lock"))?; + let fingerprint = compute_deps_fingerprint(&[], &[], &CargoFeatureSelection::default(), Some(project_root)); + let incan_lock = IncanLock::new(fingerprint, CargoFeatureSelection::default(), cargo_lock_payload); + incan_lock.write(&project_root.join("incan.lock"))?; + + let entry_path = scripts_dir.join("check.incn"); + let output_dir = project_root.join("target").join("incan").join("check"); + let entry_arg = entry_path + .to_str() + .ok_or("entry path should be valid utf-8 for prepare_project test")?; + let output_arg = output_dir + .to_str() + .ok_or("output path should be valid utf-8 for prepare_project test")?; + + prepare_project( + entry_arg, + Some(output_arg), + &CargoPolicy::default(), + Vec::new(), + false, + false, + )?; + + let generated_manifest = std::fs::read_to_string(output_dir.join("Cargo.toml"))?; + assert!( + !generated_manifest.contains("datafusion"), + "unused package-level rust dependencies should not be emitted for a script run:\n{generated_manifest}" + ); + Ok(()) + } + #[cfg(feature = "rust_inspect")] #[test] fn library_rust_abi_query_paths_include_rust_extern_backing_items() -> Result<(), Box> { diff --git a/src/cli/commands/common.rs b/src/cli/commands/common.rs index 597c0cc7e..9e19dde2e 100644 --- a/src/cli/commands/common.rs +++ b/src/cli/commands/common.rs @@ -31,7 +31,7 @@ use crate::project_lifecycle::toolchain::ToolchainConstraintSet; #[cfg(feature = "rust_inspect")] use crate::rust_inspect::{Inspector, InspectorConfig}; use incan_core::lang::{ - stdlib::{self, StdlibExtraCrateSource}, + stdlib::{self, StdlibExtraCrateDep, StdlibExtraCrateSource}, surface::result_methods, }; #[cfg(feature = "rust_inspect")] @@ -327,30 +327,7 @@ pub(crate) fn collect_project_requirements( continue; }; for dep in namespace.extra_crate_deps { - let spec = match dep.source { - StdlibExtraCrateSource::Version(version) => DependencySpec { - crate_name: dep.crate_name.to_string(), - version: Some(version.to_string()), - features: vec![], - default_features: true, - source: DependencySource::Registry, - optional: false, - package: None, - }, - StdlibExtraCrateSource::Path(relative_path) => DependencySpec { - crate_name: dep.crate_name.to_string(), - version: None, - features: vec![], - default_features: true, - source: DependencySource::Path { - path: workspace_root.join(relative_path), - }, - optional: false, - package: None, - }, - } - .normalized(); - + let spec = dependency_spec_from_stdlib_dep(dep, &workspace_root); merge_requirement_dependency( &mut requirements.dependencies, spec, @@ -361,16 +338,7 @@ pub(crate) fn collect_project_requirements( let needs_serde_runtime = needs_legacy_serde_runtime || stdlib_namespaces.contains("serde"); if needs_serde_runtime { - let serde = DependencySpec { - crate_name: "serde".to_string(), - version: Some("1.0".to_string()), - features: vec!["derive".to_string()], - default_features: true, - source: DependencySource::Registry, - optional: false, - package: None, - } - .normalized(); + let serde = dependency_spec_from_stdlib_extra_crate("serde")?; merge_requirement_dependency( &mut requirements.dependencies, serde, @@ -399,6 +367,42 @@ pub(crate) fn collect_project_requirements( Ok(requirements) } +fn dependency_spec_from_stdlib_extra_crate(crate_name: &str) -> CliResult { + let dep = stdlib::find_extra_crate_dep(crate_name).ok_or_else(|| { + CliError::failure(format!( + "stdlib dependency metadata for `{crate_name}` is missing from the registry" + )) + })?; + let workspace_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + Ok(dependency_spec_from_stdlib_dep(dep, &workspace_root)) +} + +fn dependency_spec_from_stdlib_dep(dep: &StdlibExtraCrateDep, workspace_root: &Path) -> DependencySpec { + match dep.source { + StdlibExtraCrateSource::Version(version) => DependencySpec { + crate_name: dep.crate_name.to_string(), + version: Some(version.to_string()), + features: dep.features.iter().map(|feature| (*feature).to_string()).collect(), + default_features: true, + source: DependencySource::Registry, + optional: false, + package: stdlib::extra_crate_package_alias(dep.crate_name).map(str::to_string), + }, + StdlibExtraCrateSource::Path(relative_path) => DependencySpec { + crate_name: dep.crate_name.to_string(), + version: None, + features: dep.features.iter().map(|feature| (*feature).to_string()).collect(), + default_features: true, + source: DependencySource::Path { + path: workspace_root.join(relative_path), + }, + optional: false, + package: None, + }, + } + .normalized() +} + /// Merge a dependency requirement into a collection of requirements. /// /// Existing entries with the same crate name must be compatible. @@ -463,20 +467,141 @@ pub(crate) fn merge_project_requirement_dependencies( Ok(()) } +pub(crate) fn merge_project_requirements( + current: &ProjectRequirements, + extra: &ProjectRequirements, +) -> CliResult { + let stdlib_features = current + .stdlib_features + .iter() + .chain(extra.stdlib_features.iter()) + .cloned() + .collect::>() + .into_iter() + .collect(); + + let mut dependencies = current.dependencies.clone(); + for candidate in &extra.dependencies { + if let Some(existing) = dependencies.iter().find(|dep| dep.crate_name == candidate.crate_name) { + if existing != candidate { + return Err(CliError::failure(format!( + "dependency requirement `{}` conflicts between project requirement contexts", + candidate.crate_name + ))); + } + continue; + } + dependencies.push(candidate.clone()); + } + dependencies.sort_by(|left, right| left.crate_name.cmp(&right.crate_name)); + + Ok(ProjectRequirements { + stdlib_features, + dependencies, + }) +} + +pub(crate) fn merge_resolved_dependencies( + current: &ResolvedDependencies, + extra: &ResolvedDependencies, +) -> CliResult { + let mut merged = current.clone(); + for candidate in &extra.dependencies { + merge_resolved_dependency(&mut merged.dependencies, &mut merged.dev_dependencies, candidate, false)?; + } + for candidate in &extra.dev_dependencies { + merge_resolved_dependency(&mut merged.dependencies, &mut merged.dev_dependencies, candidate, true)?; + } + merged + .dependencies + .sort_by(|left, right| left.crate_name.cmp(&right.crate_name)); + merged + .dev_dependencies + .sort_by(|left, right| left.crate_name.cmp(&right.crate_name)); + Ok(merged) +} + +fn merge_resolved_dependency( + dependencies: &mut Vec, + dev_dependencies: &mut Vec, + candidate: &DependencySpec, + dev_only: bool, +) -> CliResult<()> { + if let Some(existing) = dependencies.iter().find(|dep| dep.crate_name == candidate.crate_name) { + if existing != candidate { + return Err(CliError::failure(format!( + "dependency `{}` conflicts between resolved dependency contexts", + candidate.crate_name + ))); + } + return Ok(()); + } + + if dev_only { + if let Some(existing) = dev_dependencies + .iter() + .find(|dep| dep.crate_name == candidate.crate_name) + { + if existing != candidate { + return Err(CliError::failure(format!( + "dev dependency `{}` conflicts between resolved dependency contexts", + candidate.crate_name + ))); + } + return Ok(()); + } + dev_dependencies.push(candidate.clone()); + return Ok(()); + } + + if let Some(existing_idx) = dev_dependencies + .iter() + .position(|dep| dep.crate_name == candidate.crate_name) + { + if dev_dependencies[existing_idx] != *candidate { + return Err(CliError::failure(format!( + "dependency `{}` conflicts between dependency and dev-dependency contexts", + candidate.crate_name + ))); + } + dev_dependencies.remove(existing_idx); + } + dependencies.push(candidate.clone()); + Ok(()) +} + #[cfg(feature = "rust_inspect")] const RUST_INSPECT_WORKSPACE_FINGERPRINT_FILE: &str = ".incan_rust_inspect_fingerprint"; #[cfg(feature = "rust_inspect")] const RUST_INSPECT_WORKSPACE_FINGERPRINT_PREFIX: &str = "v1:"; -/// Counts how many times the rust-inspect stub workspace is fully regenerated (not skipped via fingerprint). -/// Used by unit tests in this module; serialized with [`RUST_INSPECT_WORKSPACE_TEST_LOCK`]. +/// Counts how many times each rust-inspect stub workspace is fully regenerated instead of skipped via fingerprint. +/// +/// Full lib tests run in parallel and other tests can legitimately create unrelated rust-inspect workspaces, so this +/// instrumentation is keyed by generated workspace path instead of using one process-wide counter. #[cfg(all(test, feature = "rust_inspect"))] -pub(crate) static TEST_RUST_INSPECT_WORKSPACE_GENERATIONS: std::sync::atomic::AtomicU64 = - std::sync::atomic::AtomicU64::new(0); +static TEST_RUST_INSPECT_WORKSPACE_GENERATIONS: std::sync::LazyLock< + std::sync::Mutex>, +> = std::sync::LazyLock::new(|| std::sync::Mutex::new(std::collections::BTreeMap::new())); +/// Records a full rust-inspect workspace regeneration for the generated workspace path under test. #[cfg(all(test, feature = "rust_inspect"))] -static RUST_INSPECT_WORKSPACE_TEST_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(()); +fn record_test_rust_inspect_workspace_generation(workspace_dir: &Path) { + let mut counts = TEST_RUST_INSPECT_WORKSPACE_GENERATIONS + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + *counts.entry(workspace_dir.to_path_buf()).or_default() += 1; +} + +/// Returns the number of full rust-inspect workspace regenerations recorded for a generated workspace path. +#[cfg(all(test, feature = "rust_inspect"))] +fn test_rust_inspect_workspace_generations(workspace_dir: &Path) -> u64 { + let counts = TEST_RUST_INSPECT_WORKSPACE_GENERATIONS + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + counts.get(workspace_dir).copied().unwrap_or(0) +} #[cfg(feature = "rust_inspect")] fn normalized_stdlib_features_for_rust_inspect_fingerprint(features: &[String]) -> Vec { @@ -553,7 +678,7 @@ fn hash_dependency_spec_for_rust_inspect(hasher: &mut Sha256, spec: &DependencyS hasher.update(b"|dep|\0"); } -/// Stable fingerprint for inputs that define the generated rust-inspect Cargo workspace under `target/incan_lock`. +/// Stable fingerprint for inputs that define one generated rust-inspect Cargo workspace. #[cfg(feature = "rust_inspect")] fn rust_inspect_workspace_fingerprint( project_name: &str, @@ -621,14 +746,42 @@ fn rust_inspect_workspace_fingerprint( ) } +#[cfg(feature = "rust_inspect")] +fn rust_inspect_workspace_dir(project_root: &Path, project_name: &str, fingerprint: &str) -> PathBuf { + let mut safe_name = project_name + .chars() + .map(|ch| { + if ch.is_ascii_alphanumeric() || ch == '-' || ch == '_' { + ch + } else { + '_' + } + }) + .collect::(); + if safe_name.is_empty() { + safe_name.push_str("project"); + } + let suffix = fingerprint + .rsplit_once(':') + .map(|(_, hash)| hash) + .unwrap_or(fingerprint) + .chars() + .take(16) + .collect::(); + project_root + .join("target") + .join("incan_lock") + .join("rust_inspect") + .join(format!("{safe_name}-{suffix}")) +} + /// Generate the rust-inspect workspace that semantic Rust extraction should query for this project. /// /// The generated workspace intentionally uses the Rust import spelling for dependency keys, while preserving the /// published Cargo package name separately when the two differ. /// /// When the same inputs are seen again (for example across multiple `incan test` cases in one package), regeneration is -/// skipped if `target/incan_lock/.incan_rust_inspect_fingerprint` matches the computed digest and expected artifacts -/// exist. +/// skipped if the namespaced workspace fingerprint matches the computed digest and expected artifacts exist. #[cfg(feature = "rust_inspect")] pub(crate) fn ensure_rust_inspect_workspace( project_root: &Path, @@ -638,16 +791,6 @@ pub(crate) fn ensure_rust_inspect_workspace( project_requirements: &ProjectRequirements, cargo_lock_payload: Option, ) -> CliResult { - let base_rust_inspect_manifest_dir = project_root.join("target").join("incan_lock"); - let rust_inspect_manifest_dir = if project_name.starts_with("incan_cmd_") { - base_rust_inspect_manifest_dir.join(project_name) - } else { - base_rust_inspect_manifest_dir - }; - let fingerprint_path = rust_inspect_manifest_dir.join(RUST_INSPECT_WORKSPACE_FINGERPRINT_FILE); - let cargo_toml_path = rust_inspect_manifest_dir.join("Cargo.toml"); - let main_rs_path = rust_inspect_manifest_dir.join("src").join("main.rs"); - let fingerprint = rust_inspect_workspace_fingerprint( project_name, rust_edition.as_deref(), @@ -655,6 +798,10 @@ pub(crate) fn ensure_rust_inspect_workspace( &project_requirements.stdlib_features, cargo_lock_payload.as_deref(), ); + let rust_inspect_manifest_dir = rust_inspect_workspace_dir(project_root, project_name, &fingerprint); + let fingerprint_path = rust_inspect_manifest_dir.join(RUST_INSPECT_WORKSPACE_FINGERPRINT_FILE); + let cargo_toml_path = rust_inspect_manifest_dir.join("Cargo.toml"); + let main_rs_path = rust_inspect_manifest_dir.join("src").join("main.rs"); let fingerprint_matches = match fs::read_to_string(&fingerprint_path) { Ok(existing) => existing.trim() == fingerprint.as_str(), @@ -683,7 +830,7 @@ pub(crate) fn ensure_rust_inspect_workspace( rust_inspect_stub.push_str("fn main() {}"); #[cfg(all(test, feature = "rust_inspect"))] - TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + record_test_rust_inspect_workspace_generation(&rust_inspect_manifest_dir); generator.generate(rust_inspect_stub.as_str()).map_err(|e| { CliError::failure(format!( @@ -1051,7 +1198,7 @@ pub fn collect_modules(entry_path: &str) -> CliResult> { /// This explicit sort guarantees each module appears only after its direct and transitive dependencies for acyclic /// portions of the graph. For cyclic components (for example stdlib prelude re-export loops), we keep deterministic /// fallback ordering rather than hard-failing in collection. -fn topologically_sort_modules( +pub(crate) fn topologically_sort_modules( modules: Vec, dependency_edges: &HashMap>, ) -> CliResult> { @@ -1310,6 +1457,31 @@ pub(crate) fn collect_inline_rust_imports(module: &ParsedModule, is_test_context imports } +/// Extract all Rust dependency uses from a parsed module. +pub(crate) fn collect_rust_dependency_uses(module: &ParsedModule, is_test_context: bool) -> Vec { + let mut imports = collect_inline_rust_imports(module, is_test_context); + let Some(rust_module_path) = &module.ast.rust_module_path else { + return imports; + }; + let Some(crate_name) = rust_module_path.node.split("::").next().filter(|name| !name.is_empty()) else { + return imports; + }; + if crate_name == stdlib::STDLIB_ROOT || stdlib::is_path_extra_crate_dep(crate_name) { + return imports; + } + + imports.push(build_inline_rust_import( + crate_name, + format!("rust.module(\"{}\")", rust_module_path.node), + &None, + &[], + rust_module_path.span, + &module.file_path, + is_test_context, + )); + imports +} + /// Build a map of file paths to source contents for error reporting. pub(crate) fn build_source_map(modules: &[ParsedModule]) -> HashMap { let mut sources = HashMap::new(); @@ -1550,6 +1722,18 @@ mod tests { }) } + fn registry_dependency(crate_name: &str, version: &str) -> DependencySpec { + DependencySpec { + crate_name: crate_name.to_string(), + version: Some(version.to_string()), + features: Vec::new(), + default_features: true, + source: DependencySource::Registry, + optional: false, + package: None, + } + } + fn write_minimal_library_artifact( root: &Path, dependency_key: &str, @@ -1567,6 +1751,80 @@ mod tests { Ok(()) } + #[test] + fn collect_rust_dependency_uses_includes_rust_module_root() -> Result<(), Box> { + let module = parsed_module_for_test("rust.module(\"datafusion::prelude\")\n\ndef main() -> None:\n pass\n")?; + + let imports = collect_rust_dependency_uses(&module, false); + + assert!( + imports.iter().any(|import| import.crate_name == "datafusion" + && import.import_path == "rust.module(\"datafusion::prelude\")"), + "rust.module roots should participate in dependency resolution: {imports:?}" + ); + Ok(()) + } + + #[test] + fn collect_rust_dependency_uses_skips_stdlib_path_extra_crate_roots() -> Result<(), Box> { + let module = parsed_module_for_test("rust.module(\"incan_web_macros\")\n\ndef main() -> None:\n pass\n")?; + + let imports = collect_rust_dependency_uses(&module, false); + + assert!( + imports.iter().all(|import| import.crate_name != "incan_web_macros"), + "stdlib-managed path crates should come from project requirements, not rust.module dependency uses: {imports:?}" + ); + Ok(()) + } + + #[test] + fn merge_resolved_dependencies_unions_dependency_contexts() -> Result<(), Box> { + let current = ResolvedDependencies { + dependencies: vec![registry_dependency("serde", "1")], + dev_dependencies: vec![registry_dependency("tokio", "1")], + }; + let extra = ResolvedDependencies { + dependencies: vec![ + registry_dependency("tokio", "1"), + registry_dependency("datafusion", "53"), + ], + dev_dependencies: Vec::new(), + }; + + let merged = merge_resolved_dependencies(¤t, &extra)?; + + assert_eq!( + merged + .dependencies + .iter() + .map(|dependency| dependency.crate_name.as_str()) + .collect::>(), + vec!["datafusion", "serde", "tokio"] + ); + assert!(merged.dev_dependencies.is_empty()); + Ok(()) + } + + #[test] + fn merge_resolved_dependencies_rejects_conflicting_contexts() { + let current = ResolvedDependencies { + dependencies: vec![registry_dependency("serde", "1")], + dev_dependencies: Vec::new(), + }; + let extra = ResolvedDependencies { + dependencies: vec![registry_dependency("serde", "2")], + dev_dependencies: Vec::new(), + }; + + let error = match merge_resolved_dependencies(¤t, &extra) { + Ok(merged) => panic!("expected conflict, got merged dependencies: {merged:?}"), + Err(error) => error, + }; + assert!(error.message.contains("serde")); + assert!(error.message.contains("conflicts")); + } + #[test] fn compilation_session_parses_with_imported_library_vocab() -> Result<(), Box> { let tmp = tempfile::tempdir()?; @@ -2617,14 +2875,19 @@ pub def main() -> int: #[cfg(feature = "rust_inspect")] #[test] - fn ensure_rust_inspect_workspace_uses_rust_safe_dependency_keys() -> Result<(), Box> { - use std::sync::atomic::Ordering; + fn rust_inspect_workspace_dir_is_namespaced_by_input_fingerprint() { + let root = Path::new("/workspace"); + let first = super::rust_inspect_workspace_dir(root, "demo", "v1:aaaaaaaaaaaaaaaaaaaaaaaa"); + let second = super::rust_inspect_workspace_dir(root, "demo", "v1:bbbbbbbbbbbbbbbbbbbbbbbb"); - let _serial = super::RUST_INSPECT_WORKSPACE_TEST_LOCK - .lock() - .map_err(|e| format!("rust-inspect workspace test lock poisoned: {e}"))?; - super::TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.store(0, Ordering::SeqCst); + assert_ne!(first, second); + assert!(first.ends_with(Path::new("target/incan_lock/rust_inspect/demo-aaaaaaaaaaaaaaaa"))); + assert!(second.ends_with(Path::new("target/incan_lock/rust_inspect/demo-bbbbbbbbbbbbbbbb"))); + } + #[cfg(feature = "rust_inspect")] + #[test] + fn ensure_rust_inspect_workspace_uses_rust_safe_dependency_keys() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let requirements = ProjectRequirements::default(); let resolved = ResolvedDependencies { @@ -2649,7 +2912,7 @@ pub def main() -> int: Some("[[package]]\nname = \"metadata_probe\"\n".to_string()), )?; assert_eq!( - super::TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.load(Ordering::SeqCst), + super::test_rust_inspect_workspace_generations(&out_dir), 1, "expected one rust-inspect workspace generation" ); @@ -2680,13 +2943,6 @@ pub def main() -> int: #[cfg(feature = "rust_inspect")] #[test] fn ensure_rust_inspect_workspace_skips_regeneration_when_unchanged() -> Result<(), Box> { - use std::sync::atomic::Ordering; - - let _serial = super::RUST_INSPECT_WORKSPACE_TEST_LOCK - .lock() - .map_err(|e| format!("rust-inspect workspace test lock poisoned: {e}"))?; - super::TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.store(0, Ordering::SeqCst); - let tmp = tempfile::tempdir()?; let requirements = ProjectRequirements::default(); let resolved = ResolvedDependencies { @@ -2703,7 +2959,7 @@ pub def main() -> int: }; let lock = Some("[[package]]\nname = \"skip_probe\"\n".to_string()); - ensure_rust_inspect_workspace( + let out_dir = ensure_rust_inspect_workspace( tmp.path(), "skip_probe", Some("2021".to_string()), @@ -2712,7 +2968,7 @@ pub def main() -> int: lock.clone(), )?; assert_eq!( - super::TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.load(Ordering::SeqCst), + super::test_rust_inspect_workspace_generations(&out_dir), 1, "first call should generate the workspace" ); @@ -2726,7 +2982,7 @@ pub def main() -> int: lock, )?; assert_eq!( - super::TEST_RUST_INSPECT_WORKSPACE_GENERATIONS.load(Ordering::SeqCst), + super::test_rust_inspect_workspace_generations(&out_dir), 1, "second call with identical inputs should skip regeneration" ); diff --git a/src/cli/commands/lock.rs b/src/cli/commands/lock.rs index 65cc81827..fdbf6bda2 100644 --- a/src/cli/commands/lock.rs +++ b/src/cli/commands/lock.rs @@ -16,16 +16,17 @@ use sha2::{Digest, Sha256}; use crate::backend::ProjectGenerator; use crate::cli::prelude::ParsedModule; use crate::cli::{CliError, CliResult, ExitCode}; -use crate::dependency_resolver::{InlineRustImport, ResolvedDependencies, resolve_dependencies}; +use crate::dependency_resolver::{InlineRustImport, ResolvedDependencies, resolve_reachable_dependencies}; use crate::frontend::library_manifest_index::LibraryManifestIndex; use crate::frontend::{diagnostics, lexer, parser}; use crate::lockfile::{CargoFeatureSelection, IncanLock, compute_deps_fingerprint}; use crate::manifest::ProjectManifest; use super::common::{ - CargoPolicy, ProjectRequirements, build_source_map, cargo_command_flags, cargo_lockfile_flags, - collect_inline_rust_imports, collect_modules, collect_project_requirements, enforce_project_toolchain_constraint, - format_dependency_error, merge_project_requirement_dependencies, + CargoPolicy, ProjectRequirements, build_source_map, cargo_command_flags, cargo_lockfile_flags, collect_modules, + collect_project_requirements, collect_rust_dependency_uses, enforce_project_toolchain_constraint, + format_dependency_error, merge_project_requirement_dependencies, merge_project_requirements, + merge_resolved_dependencies, }; #[cfg(feature = "rust_inspect")] use super::common::{collect_rust_inspect_query_paths, ensure_rust_inspect_workspace, prewarm_rust_inspect_workspace}; @@ -135,11 +136,18 @@ pub(crate) fn resolve_lock_payload(request: LockResolutionRequest<'_>) -> CliRes } else { None }; - let (resolved, project_requirements) = if let Some(context) = project_context.as_ref() { - (&context.resolved, &context.project_requirements) + let lock_inputs = if let Some(context) = project_context.as_ref() { + Some(( + merge_resolved_dependencies(resolved, &context.resolved)?, + merge_project_requirements(project_requirements, &context.project_requirements)?, + )) } else { - (resolved, project_requirements) + None }; + let (resolved, project_requirements) = lock_inputs + .as_ref() + .map(|(resolved, requirements)| (resolved, requirements)) + .unwrap_or((resolved, project_requirements)); #[cfg(feature = "rust_inspect")] let rust_inspect_query_paths = project_context .as_ref() @@ -269,12 +277,12 @@ fn collect_project_lock_context( let mut inline_imports = Vec::new(); for module in &modules { - inline_imports.extend(collect_inline_rust_imports(module, false)); + inline_imports.extend(collect_rust_dependency_uses(module, false)); } inline_imports.extend(test_inputs.inline_imports); let mut resolved = - resolve_dependencies(Some(manifest), &inline_imports, true, cargo_features).map_err(|errors| { + resolve_reachable_dependencies(Some(manifest), &inline_imports, true, cargo_features).map_err(|errors| { let mut msg = String::new(); let sources = build_source_map(&project_requirement_modules); for err in errors { @@ -674,7 +682,7 @@ fn collect_test_lock_inputs( source: source.clone(), ast: ast.clone(), }; - inline_imports.extend(collect_inline_rust_imports(&test_module, true)); + inline_imports.extend(collect_rust_dependency_uses(&test_module, true)); project_requirement_modules.push(test_module); let source_modules = crate::cli::test_runner::collect_source_modules_for_test( @@ -686,7 +694,7 @@ fn collect_test_lock_inputs( ) .map_err(CliError::failure)?; for module in &source_modules { - inline_imports.extend(collect_inline_rust_imports(module, false)); + inline_imports.extend(collect_rust_dependency_uses(module, false)); } project_requirement_modules.extend(source_modules); } diff --git a/src/cli/commands/tools.rs b/src/cli/commands/tools.rs index e801aff24..dca8c176d 100644 --- a/src/cli/commands/tools.rs +++ b/src/cli/commands/tools.rs @@ -13,7 +13,8 @@ use crate::cli::prelude::ParsedModule; use crate::cli::{CliError, CliResult, ExitCode}; use crate::frontend::api_metadata::{ ApiDeclaration, ApiFunction, ApiPartial, CHECKED_API_METADATA_SCHEMA_VERSION, CheckedApiMetadataPackage, - CheckedApiPackageIdentity, collect_checked_api_metadata, validate_checked_api_docstrings, + CheckedApiPackageIdentity, collect_checked_api_metadata, materialize_api_alias_projections, + validate_checked_api_docstrings, }; use crate::frontend::contract_metadata::{ CanonicalModelBundle, read_model_bundles_from_json, read_project_model_bundles, @@ -417,6 +418,8 @@ fn collect_api_metadata_package(path: &Path) -> CliResult Result<(), Box> { + let tmp = tempfile::tempdir()?; + let src = tmp.path().join("src"); + let operators = src.join("functions").join("operators"); + fs::create_dir_all(&operators)?; + fs::write( + tmp.path().join("incan.toml"), + r#" +[project] +name = "metadata_registry" +version = "0.1.0" +"#, + )?; + fs::write( + src.join("registry.incn"), + r#" +pub def registered[F](spec: str) -> ((F) -> F): + return (func) => func +"#, + )?; + fs::write( + operators.join("eq.incn"), + r#" +from registry import registered + +pub model ColumnExpr: + pub name: str + +@registered("equal") +pub def eq(left: ColumnExpr, right: ColumnExpr) -> ColumnExpr: + return left +"#, + )?; + fs::write( + operators.join("mod.incn"), + "pub from functions.operators.eq import eq\n", + )?; + fs::write(src.join("lib.incn"), "pub from functions.operators.mod import eq\n")?; + + let package = collect_api_metadata_package(tmp.path())?; + let lib_alias = package + .modules + .iter() + .find(|module| module.module_path == vec!["lib".to_string()]) + .and_then(|module| { + module.declarations.iter().find_map(|decl| match decl { + ApiDeclaration::Alias(alias) if alias.name == "eq" => Some(alias), + _ => None, + }) + }) + .ok_or("expected lib facade alias")?; + let projection = lib_alias + .projected_function + .as_ref() + .ok_or("expected projected function metadata on facade alias")?; + + assert_eq!(projection.callable.name, "eq"); + assert_eq!( + projection.source_path, + vec![ + "functions".to_string(), + "operators".to_string(), + "eq".to_string(), + "eq".to_string(), + ] + ); + assert_eq!( + projection + .callable + .params + .iter() + .map(|param| param.name.as_str()) + .collect::>(), + vec!["left", "right"] + ); + assert!( + projection.decorators.iter().any(|decorator| { + decorator.path == vec!["registry".to_string(), "registered".to_string()] + && decorator + .decorated_callable + .as_ref() + .is_some_and(|callable| callable.name == "eq") + }), + "expected projected decorator metadata with decorated callable context, got {projection:?}" + ); + Ok(()) + } + #[test] fn cargo_config_hints_detect_vendor_source_replacement() -> Result<(), Box> { let tmp = tempfile::tempdir()?; diff --git a/src/cli/test_runner/execution.rs b/src/cli/test_runner/execution.rs index 5e6cf26ed..1716829bd 100644 --- a/src/cli/test_runner/execution.rs +++ b/src/cli/test_runner/execution.rs @@ -15,7 +15,7 @@ use crate::cli::commands::common::{ collect_rust_inspect_query_paths, ensure_rust_inspect_workspace, prewarm_rust_inspect_workspace, }; use crate::cli::prelude::ParsedModule; -use crate::dependency_resolver::resolve_dependencies; +use crate::dependency_resolver::resolve_reachable_dependencies; use crate::dependency_resolver::{InlineRustImport, ResolvedDependencies}; use crate::frontend::ast::{ AssertKind, AssertStmt, CallArg, Declaration, DictEntry, Expr, ImportItem, ImportKind, ListEntry, ParamKind, @@ -23,6 +23,7 @@ use crate::frontend::ast::{ }; use crate::frontend::decorator_resolution; use crate::frontend::library_manifest_index::LibraryManifestIndex; +use crate::frontend::module::logical_module_segments_from_file; use crate::frontend::testing_markers::{TestingMarkerKind, load_testing_marker_semantics, resolve_testing_marker_kind}; use crate::frontend::vocab_desugar_pass; use crate::frontend::{lexer, parser}; @@ -33,7 +34,7 @@ use sha2::{Digest, Sha256}; use super::module_graph::collect_source_modules_for_test; use super::types::{FixtureScope, TestInfo, TestResult}; -/// Generated `#[cfg(test)]` module that wraps Incan test functions as Rust `#[test]` cases, one `cargo test` per file. +/// Generated `#[cfg(test)]` module that wraps Incan test functions as Rust `#[test]` cases. const INCAN_FILE_TEST_MOD: &str = "__incan_file_tests"; #[derive(Debug, Clone, Copy, Default)] @@ -67,9 +68,9 @@ fn collect_test_dependency_inline_imports( test_module: &ParsedModule, source_modules: &[ParsedModule], ) -> Vec { - let mut inline_imports = common::collect_inline_rust_imports(test_module, true); + let mut inline_imports = common::collect_rust_dependency_uses(test_module, true); for module in source_modules { - inline_imports.extend(common::collect_inline_rust_imports(module, false)); + inline_imports.extend(common::collect_rust_dependency_uses(module, false)); } inline_imports } @@ -219,14 +220,27 @@ fn dedupe_import_declarations(ast: &mut Program) { ast.declarations = declarations; } -#[derive(Debug, Default)] +#[derive(Debug, Clone, Default)] struct TopLevelNames { types: HashSet, values: HashSet, + imported_types: HashSet, + imported_values: HashSet, +} + +#[derive(Debug, Clone)] +struct TopLevelNameSummary { + path: PathBuf, + names: TopLevelNames, } /// Collect top-level Rust item names that would collide if multiple Incan files were concatenated. fn collect_top_level_decl_names(program: &Program) -> TopLevelNames { + fn add_import_binding(name: &str, names: &mut TopLevelNames) { + names.imported_types.insert(name.to_string()); + names.imported_values.insert(name.to_string()); + } + /// Add the Rust type/value namespace names contributed by one declaration. fn collect_from_decl(decl: &Declaration, names: &mut TopLevelNames) { match decl { @@ -268,7 +282,40 @@ fn collect_top_level_decl_names(program: &Program) -> TopLevelNames { collect_from_decl(&nested.node, names); } } - Declaration::Import(_) | Declaration::Partial(_) | Declaration::Docstring(_) => {} + Declaration::Import(decl) => match &decl.kind { + ImportKind::Module(path) => { + let local = decl + .alias + .as_ref() + .or_else(|| path.segments.last()) + .map(String::as_str) + .unwrap_or("module"); + add_import_binding(local, names); + } + ImportKind::From { items, .. } + | ImportKind::PubFrom { items, .. } + | ImportKind::RustFrom { items, .. } => { + for item in items { + add_import_binding(item.alias.as_deref().unwrap_or(&item.name), names); + } + } + ImportKind::PubLibrary { library } => { + add_import_binding(decl.alias.as_deref().unwrap_or(library), names); + } + ImportKind::Python(pkg) => { + add_import_binding(decl.alias.as_deref().unwrap_or(pkg), names); + } + ImportKind::RustCrate { crate_name, path, .. } => { + let local = decl + .alias + .as_ref() + .or_else(|| path.last()) + .map(String::as_str) + .unwrap_or(crate_name); + add_import_binding(local, names); + } + }, + Declaration::Partial(_) | Declaration::Docstring(_) => {} } } @@ -279,51 +326,102 @@ fn collect_top_level_decl_names(program: &Program) -> TopLevelNames { names } -/// Return whether concatenating source files into one worker harness would collide at Rust module scope. -/// -/// Worker batches can share one process only when their source files can coexist in the generated crate. If two files -/// define the same model, function, or other top-level Rust item, the runner falls back to per-file harnesses. -fn batch_has_cross_file_top_level_collision( +fn collect_top_level_name_summary( + path: &Path, + source: &str, + library_imported_vocab: Option<&parser::ImportedLibraryVocab>, +) -> Option { + let tokens = lexer::lex(source).ok()?; + let ast = + parser::parse_with_context(&tokens, Some(path.to_string_lossy().as_ref()), library_imported_vocab).ok()?; + let names = collect_top_level_decl_names(&ast_with_inline_test_declarations(&ast)); + Some(TopLevelNameSummary { + path: path.to_path_buf(), + names, + }) +} + +fn collect_top_level_name_summaries( sources_by_file: &[(PathBuf, String)], library_imported_vocab: Option<&parser::ImportedLibraryVocab>, -) -> bool { - if sources_by_file.len() <= 1 { - return false; - } +) -> Option> { + sources_by_file + .iter() + .map(|(path, source)| collect_top_level_name_summary(path, source, library_imported_vocab)) + .collect() +} +fn top_level_summaries_have_collision<'a>(summaries: impl IntoIterator) -> bool { let mut type_owner: HashMap = HashMap::new(); let mut value_owner: HashMap = HashMap::new(); - for (path, source) in sources_by_file { - let Ok(tokens) = lexer::lex(source) else { - return false; - }; - let Ok(ast) = - parser::parse_with_context(&tokens, Some(path.to_string_lossy().as_ref()), library_imported_vocab) - else { - return false; - }; - let names = collect_top_level_decl_names(&ast_with_inline_test_declarations(&ast)); - for name in names.types { + let mut imported_type_owner: HashMap = HashMap::new(); + let mut imported_value_owner: HashMap = HashMap::new(); + for summary in summaries { + for name in &summary.names.types { + if imported_type_owner + .get(name) + .is_some_and(|owner| owner != &summary.path) + { + return true; + } if type_owner - .insert(name, path.clone()) - .is_some_and(|owner| owner != *path) + .insert(name.clone(), summary.path.clone()) + .is_some_and(|owner| owner != summary.path) { return true; } } - for name in names.values { + for name in &summary.names.values { + if imported_value_owner + .get(name) + .is_some_and(|owner| owner != &summary.path) + { + return true; + } if value_owner - .insert(name, path.clone()) - .is_some_and(|owner| owner != *path) + .insert(name.clone(), summary.path.clone()) + .is_some_and(|owner| owner != summary.path) { return true; } } + for name in &summary.names.imported_types { + if type_owner.get(name).is_some_and(|owner| owner != &summary.path) { + return true; + } + imported_type_owner + .entry(name.clone()) + .or_insert_with(|| summary.path.clone()); + } + for name in &summary.names.imported_values { + if value_owner.get(name).is_some_and(|owner| owner != &summary.path) { + return true; + } + imported_value_owner + .entry(name.clone()) + .or_insert_with(|| summary.path.clone()); + } } false } +/// Return whether concatenating source files into one worker harness would collide at Rust module scope. +/// +/// Worker batches can share one process only when their source files can coexist in the generated crate. If two files +/// define the same model, function, or another top-level Rust item, or when one file imports a name another file +/// declares, the runner falls back to per-file harnesses. +fn batch_has_cross_file_top_level_collision( + sources_by_file: &[(PathBuf, String)], + library_imported_vocab: Option<&parser::ImportedLibraryVocab>, +) -> bool { + if sources_by_file.len() <= 1 { + return false; + } + collect_top_level_name_summaries(sources_by_file, library_imported_vocab) + .is_some_and(|summaries| top_level_summaries_have_collision(&summaries)) +} + /// Partition files into greedy groups that can still share a generated Rust module scope. /// /// A single duplicate top-level name should not force the whole worker batch back to one Cargo harness per file. @@ -333,25 +431,281 @@ fn partition_collision_free_file_groups( sources_by_file: &[(PathBuf, String)], library_imported_vocab: Option<&parser::ImportedLibraryVocab>, ) -> Vec> { - let mut groups: Vec> = Vec::new(); - 'source: for (path, source) in sources_by_file { + let Some(summaries) = collect_top_level_name_summaries(sources_by_file, library_imported_vocab) else { + return vec![sources_by_file.iter().map(|(path, _)| path.clone()).collect()]; + }; + + let mut groups: Vec> = Vec::new(); + 'source: for summary in summaries { for group in &mut groups { let mut candidate = group.clone(); - candidate.push((path.clone(), source.clone())); - if !batch_has_cross_file_top_level_collision(&candidate, library_imported_vocab) { - group.push((path.clone(), source.clone())); + candidate.push(summary.clone()); + if !top_level_summaries_have_collision(&candidate) { + group.push(summary); continue 'source; } } - groups.push(vec![(path.clone(), source.clone())]); + groups.push(vec![summary]); } groups .into_iter() - .map(|group| group.into_iter().map(|(path, _)| path).collect()) + .map(|group| group.into_iter().map(|summary| summary.path).collect()) .collect() } +fn rebase_token_spans(tokens: &mut [lexer::Token], source_offset: usize) { + if source_offset == 0 { + return; + } + + for token in tokens { + token.span.start = token.span.start.saturating_add(source_offset); + token.span.end = token.span.end.saturating_add(source_offset); + if let lexer::TokenKind::FString(parts) = &mut token.kind { + for part in parts { + if let lexer::FStringPart::Expr { offset, .. } = part { + *offset = offset.saturating_add(source_offset); + } + } + } + } +} + +/// Parse each source file in a generated test batch independently, then merge declarations for the shared harness. +/// +/// The parser's `module tests:` cardinality rule is intentionally per source file. A worker batch may contain several +/// files, so the runner must not concatenate source text and ask the parser to treat that batch as one file. +fn parse_test_batch_sources( + batch_sources: &[(PathBuf, String)], + library_imported_vocab: Option<&parser::ImportedLibraryVocab>, + library_imported_dsl_surfaces: Option<&parser::ImportedLibraryDslSurfaces>, +) -> Result { + let mut declarations = Vec::new(); + let mut warnings = Vec::new(); + let mut rust_module_path = None; + let mut source_offset = 0usize; + let source_path = batch_sources + .first() + .map(|(path, _)| path.to_string_lossy().to_string()); + + for (path, source) in batch_sources { + let mut tokens = lexer::lex(source).map_err(|e| format!("Lexer error in {}: {:?}", path.display(), e))?; + rebase_token_spans(&mut tokens, source_offset); + let parsed = parser::parse_with_context_and_surfaces( + &tokens, + Some(path.to_string_lossy().as_ref()), + library_imported_vocab, + library_imported_dsl_surfaces, + ) + .map_err(|e| format!("Parser error in {}: {:?}", path.display(), e))?; + if let Some(module_path) = parsed.rust_module_path { + if rust_module_path.is_some() { + return Err(format!( + "Parser error in {}: duplicate rust.module() directives in test batch", + path.display() + )); + } + rust_module_path = Some(module_path); + } + warnings.extend(parsed.warnings); + declarations.extend(parsed.declarations); + source_offset = source_offset.saturating_add(source.len()).saturating_add(1); + } + + Ok(Program { + declarations, + source_path, + rust_module_path, + warnings, + }) +} + +struct InlineSourceModuleBatch { + ast: Program, + source_modules: Vec, + harnesses: Vec, +} + +fn empty_test_batch_root(first_path: &Path) -> Program { + Program { + declarations: Vec::new(), + source_path: Some(first_path.to_string_lossy().to_string()), + rust_module_path: None, + warnings: Vec::new(), + } +} + +fn program_has_inline_test_module(program: &Program) -> bool { + program + .declarations + .iter() + .any(|decl| matches!(decl.node, Declaration::TestModule(_))) +} + +fn prepare_runner_program(ast: &Program) -> Result<(Program, HashMap), String> { + let mut runner_ast = ast_with_inline_test_declarations(ast); + normalize_runner_assert_statements(&mut runner_ast); + prune_shadowed_fixture_declarations(&mut runner_ast); + dedupe_import_declarations(&mut runner_ast); + let mut fixtures = collect_fixture_execution_info(&runner_ast, &HashMap::new()); + let fixture_teardowns = split_yield_fixture_declarations(&mut runner_ast)?; + apply_fixture_teardowns(&mut fixtures, &fixture_teardowns); + Ok((runner_ast, fixtures)) +} + +fn parse_and_desugar_test_sources( + batch_sources: &[(PathBuf, String)], + library_manifest_index: &LibraryManifestIndex, + library_imported_vocab: &parser::ImportedLibraryVocab, + library_imported_dsl_surfaces: &parser::ImportedLibraryDslSurfaces, +) -> Result { + let mut ast = parse_test_batch_sources( + batch_sources, + Some(library_imported_vocab), + Some(library_imported_dsl_surfaces), + )?; + let path_display = batch_sources + .last() + .or_else(|| batch_sources.first()) + .map(|(path, _)| path.to_string_lossy()); + if let Err(errors) = + vocab_desugar_pass::desugar_program_vocab_blocks(&mut ast, path_display.as_deref(), library_manifest_index) + { + return Err(format!("Vocab desugar error: {:?}", errors)); + } + Ok(ast) +} + +fn module_name_for_segments(segments: &[String]) -> String { + let mut hasher = Sha256::new(); + for segment in segments { + hasher.update(segment.as_bytes()); + hasher.update([0]); + } + let digest = hex::encode(hasher.finalize()); + let stem = if segments.is_empty() { + "module".to_string() + } else { + segments.join("_") + }; + format!("{stem}_{}", &digest[..8]) +} + +fn read_conftest_sources(paths: &[PathBuf]) -> Result, String> { + let mut sources = Vec::new(); + for path in paths { + let source = + fs::read_to_string(path).map_err(|err| format!("Failed to read conftest {}: {}", path.display(), err))?; + sources.push((path.clone(), source)); + } + Ok(sources) +} + +fn prepare_inline_source_module_batch( + sources_by_file: &[(PathBuf, String)], + conftest_files_by_file: &HashMap>, + source_root: &Path, + library_manifest_index: &LibraryManifestIndex, + library_imported_vocab: &parser::ImportedLibraryVocab, + library_imported_dsl_surfaces: &parser::ImportedLibraryDslSurfaces, +) -> Result, String> { + if sources_by_file.len() <= 1 { + return Ok(None); + } + + let mut source_modules = Vec::new(); + let mut harnesses = Vec::new(); + let mut batch_files = HashSet::new(); + let mut seen_module_paths = HashSet::new(); + let mut parsed_sources = Vec::new(); + + for (path, source) in sources_by_file { + let Some(module_path) = logical_module_segments_from_file(source_root, path) else { + return Ok(None); + }; + let ast = parse_and_desugar_test_sources( + &[(path.clone(), source.clone())], + library_manifest_index, + library_imported_vocab, + library_imported_dsl_surfaces, + )?; + if !program_has_inline_test_module(&ast) { + return Ok(None); + } + batch_files.insert(canonical_path_for_cache_key(path)); + parsed_sources.push((path.clone(), source.clone(), module_path, ast)); + } + + let mut deferred_dependencies = Vec::new(); + for (path, source, module_path, ast) in parsed_sources { + let mut module_sources = + read_conftest_sources(conftest_files_by_file.get(&path).map(Vec::as_slice).unwrap_or(&[]))?; + module_sources.push((path.clone(), source.clone())); + let combined_ast = if module_sources.len() == 1 { + ast + } else { + parse_and_desugar_test_sources( + &module_sources, + library_manifest_index, + library_imported_vocab, + library_imported_dsl_surfaces, + )? + }; + let (runner_ast, fixtures) = prepare_runner_program(&combined_ast)?; + let module_name = module_name_for_segments(&module_path); + let module_source = module_sources + .iter() + .map(|(_, source)| source.as_str()) + .collect::>() + .join("\n"); + + for dependency in collect_source_modules_for_test( + &runner_ast, + source_root, + Some(library_imported_vocab), + Some(library_imported_dsl_surfaces), + Some(library_manifest_index), + )? { + deferred_dependencies.push(dependency); + } + + if seen_module_paths.insert(module_path.clone()) { + source_modules.push(ParsedModule { + name: module_name, + path_segments: module_path.clone(), + file_path: path.clone(), + source: module_source, + ast: runner_ast, + }); + } + harnesses.push(PreparedModuleHarness { + file_path: path, + module_path, + fixtures, + }); + } + + for dependency in deferred_dependencies { + if batch_files.contains(&canonical_path_for_cache_key(&dependency.file_path)) { + continue; + } + if seen_module_paths.insert(dependency.path_segments.clone()) { + source_modules.push(dependency); + } + } + + let first_path = sources_by_file + .first() + .map(|(path, _)| path.as_path()) + .unwrap_or_else(|| Path::new(".")); + Ok(Some(InlineSourceModuleBatch { + ast: empty_test_batch_root(first_path), + source_modules, + harnesses, + })) +} + /// Resolve a dotted expression path using local import aliases collected from the runner AST. fn resolved_expr_path(expr: &Spanned, aliases: &HashMap>) -> Option> { match &expr.node { @@ -435,8 +789,23 @@ fn normalize_runner_assert_statements(ast: &mut Program) { /// By default this reuses the project's main `target/` so existing dependency artifacts are shared across regular /// builds and `incan test` runs for better DX. /// +/// Set `INCAN_TEST_SHARED_TARGET_DIR` to force all generated test harnesses into a caller-provided target directory. +/// This is primarily useful for integration tests that create many throwaway project roots but should still reuse the +/// same compiled harness dependencies. +/// /// Set `INCAN_TEST_ISOLATED_TARGET_DIR` to one of `1|true|yes|on` to use `target/incan_test_runner` instead. fn shared_cargo_target_dir(project_root: &Path) -> PathBuf { + if let Ok(shared_target_dir) = std::env::var("INCAN_TEST_SHARED_TARGET_DIR") { + let shared_target_dir = PathBuf::from(shared_target_dir); + if shared_target_dir.is_absolute() { + return shared_target_dir; + } + if let Ok(cwd) = std::env::current_dir() { + return cwd.join(shared_target_dir); + } + return shared_target_dir; + } + let absolute_project_root = if project_root.is_absolute() { project_root.to_path_buf() } else if let Ok(cwd) = std::env::current_dir() { @@ -479,6 +848,7 @@ pub(super) struct PreparedTestFile { pub library_manifest_index: LibraryManifestIndex, pub ast: Program, pub fixtures: HashMap, + pub module_harnesses: Vec, pub source_modules: Vec, pub project_root: PathBuf, pub resolved: ResolvedDependencies, @@ -489,6 +859,13 @@ pub(super) struct PreparedTestFile { pub rust_inspect_manifest_dir: PathBuf, } +/// Runner harness metadata for one inline source file emitted as its own Rust module. +pub(super) struct PreparedModuleHarness { + pub file_path: PathBuf, + pub module_path: Vec, + pub fixtures: HashMap, +} + /// Parsed dependency context for the project lock-validation entry point, shared across test batches in one session. struct PreparedLockEntry { modules: Vec, @@ -640,7 +1017,7 @@ fn expr_references_name(expr: &Expr, name: &str) -> bool { | CallArg::KeywordUnpack(expr) => expr_references_name(&expr.node, name), }), Expr::FString(parts) => parts.iter().any(|part| { - if let crate::frontend::ast::FStringPart::Expr(expr) = part { + if let crate::frontend::ast::FStringPart::Expr { expr, .. } = part { expr_references_name(&expr.node, name) } else { false @@ -993,9 +1370,9 @@ fn compute_test_prep_cache_key( /// Merge stdlib feature flags from previously prepared files with the current file requirements. /// -/// The rust-inspect workspace lives under one shared `target/incan_lock` directory per package. If files in a single -/// `incan test` session require different stdlib features, a non-monotonic feature set can cause workspace -/// fingerprint churn and expensive mid-run rewrites. Keeping a session-local feature union avoids that churn. +/// Rust-inspect workspaces are keyed by dependency fingerprint under `target/incan_lock`. If files in a single +/// `incan test` session require different stdlib features, a non-monotonic feature set can fan out into extra +/// workspaces. Keeping a session-local feature union avoids that churn. fn merge_rust_inspect_stdlib_features<'a>( existing_feature_sets: impl Iterator, current_features: &[String], @@ -1030,7 +1407,7 @@ fn prepare_lock_entry( let modules = common::collect_modules(&lock_entry_arg).map_err(|err| err.message.clone())?; let mut inline_imports = Vec::new(); for module in &modules { - inline_imports.extend(common::collect_inline_rust_imports(module, false)); + inline_imports.extend(common::collect_rust_dependency_uses(module, false)); } let project_requirements = common::collect_project_requirements(&modules, library_manifest_index).map_err(|err| err.message.clone())?; @@ -1049,34 +1426,7 @@ fn merge_lock_project_requirements( current: &ProjectRequirements, lock_entry: &ProjectRequirements, ) -> Result { - let stdlib_features = current - .stdlib_features - .iter() - .chain(lock_entry.stdlib_features.iter()) - .cloned() - .collect::>() - .into_iter() - .collect(); - - let mut dependencies = current.dependencies.clone(); - for candidate in &lock_entry.dependencies { - if let Some(existing) = dependencies.iter().find(|dep| dep.crate_name == candidate.crate_name) { - if existing != candidate { - return Err(format!( - "dependency requirement `{}` conflicts between test batch and lock entry context", - candidate.crate_name - )); - } - continue; - } - dependencies.push(candidate.clone()); - } - dependencies.sort_by(|left, right| left.crate_name.cmp(&right.crate_name)); - - Ok(ProjectRequirements { - stdlib_features, - dependencies, - }) + common::merge_project_requirements(current, lock_entry).map_err(|err| err.message) } /// Promote project dev dependencies into ordinary dependencies for generated test-runner crates. @@ -1346,6 +1696,30 @@ fn test_runner_stdlib_features( features.into_iter().collect() } +fn test_runner_stdlib_features_for_batch( + base: &[String], + tests: &[TestInfo], + fixtures: &HashMap, + module_harnesses: &[PreparedModuleHarness], +) -> Vec { + if module_harnesses.is_empty() { + return test_runner_stdlib_features(base, tests, fixtures); + } + + let mut features = base.iter().cloned().collect::>(); + if module_harnesses.iter().any(|harness| { + let file_tests = tests + .iter() + .filter(|test| test.file_path == harness.file_path) + .cloned() + .collect::>(); + harness_needs_async_runtime(&file_tests, &harness.fixtures) + }) { + features.insert("async".to_string()); + } + features.into_iter().collect() +} + /// Generate an expression that calls a fixture, recursively filling fixture dependencies. fn fixture_arg( name: &str, @@ -1675,6 +2049,17 @@ fn inject_file_test_harness( tests: &[TestInfo], project_root: &Path, fixtures: &HashMap, +) -> String { + let test_indices = (0..tests.len()).collect::>(); + inject_file_test_harness_with_indices(rust_code, tests, &test_indices, project_root, fixtures) +} + +fn inject_file_test_harness_with_indices( + rust_code: &str, + tests: &[TestInfo], + test_indices: &[usize], + project_root: &Path, + fixtures: &HashMap, ) -> String { let mut out = rust_code.to_string(); let project_root_literal = project_root.to_string_lossy().to_string(); @@ -1750,7 +2135,7 @@ fn inject_file_test_harness( ); } let teardown_fixtures = ordered_teardown_fixtures(tests, fixtures); - for (index, t) in tests.iter().enumerate() { + for (index, t) in test_indices.iter().copied().zip(tests.iter()) { let fname = harness_fn_name(t, index); let call = harness_call(t, index, fixtures); out.push_str(" #[test]\n fn "); @@ -2323,10 +2708,11 @@ fn preheat_status_label(status: HarnessPreheatStatus) -> &'static str { } } -/// Run every collected test in `tests` that lives in the same `.incn` file with **one** `cargo test` invocation (#271). +/// Run one collected test execution unit with a single generated Cargo/libtest invocation. /// -/// Returns an empty vector when `tests` is empty. Otherwise every entry must share the same [`TestInfo::file_path`]. -/// Skip/xfail handling stays in [`super::run_tests`]. +/// Ordinary test files still use the root harness shape. Cross-file inline source batches emit each tested source file +/// as its own Rust module and inject the harness beside the file-local declarations, so imports and public declarations +/// from different source files do not share one synthetic Rust scope. #[allow(clippy::too_many_arguments)] pub(super) fn run_file_tests_batch( tests: &[TestInfo], @@ -2347,6 +2733,7 @@ pub(super) fn run_file_tests_batch( // ---- Context: load test source, discover manifest, parse and vocab-desugar the test file ---- let mut source_parts = Vec::new(); + let mut batch_parse_sources = Vec::new(); let mut sources_by_file = Vec::new(); let mut seen_conftests = BTreeSet::new(); let mut seen_files = BTreeSet::new(); @@ -2360,7 +2747,10 @@ pub(super) fn run_file_tests_batch( continue; } match fs::read_to_string(conftest) { - Ok(source) => source_parts.push(source), + Ok(source) => { + source_parts.push(source.clone()); + batch_parse_sources.push((conftest.clone(), source)); + } Err(err) => { let message = format!("Failed to read conftest {}: {}", conftest.display(), err); return tests @@ -2374,6 +2764,7 @@ pub(super) fn run_file_tests_batch( match fs::read_to_string(&test.file_path) { Ok(source) => { sources_by_file.push((test.file_path.clone(), source.clone())); + batch_parse_sources.push((test.file_path.clone(), source.clone())); source_parts.push(source); } Err(e) => { @@ -2412,84 +2803,23 @@ pub(super) fn run_file_tests_batch( let library_imported_vocab = library_manifest_index.library_imported_vocab(); let library_imported_dsl_surfaces = library_manifest_index.library_imported_dsl_surfaces(); - if batch_has_cross_file_top_level_collision(&sources_by_file, Some(&library_imported_vocab)) { - let mut split_results = Vec::new(); - for file_group in partition_collision_free_file_groups(&sources_by_file, Some(&library_imported_vocab)) { - let file_group = file_group.into_iter().collect::>(); - let file_tests = tests - .iter() - .filter(|test| file_group.contains(&test.file_path)) - .cloned() - .collect::>(); - split_results.extend(run_file_tests_batch( - &file_tests, - conftest_files_by_file, - prep_cache, - cargo_policy, - cargo_features, - cargo_no_default_features, - cargo_all_features, - options, - )); - } - return split_results; - } - - let tokens = match lexer::lex(&source) { - Ok(t) => t, - Err(e) => { - return tests - .iter() - .map(|t| { - ( - t.clone(), - TestResult::Failed(start.elapsed(), format!("Lexer error: {:?}", e)), - ) - }) - .collect(); - } - }; + // ---- Context: resolve project paths and collect transitive Incan modules for the test ---- + let project_root = manifest + .as_ref() + .map(|m| m.project_root().to_path_buf()) + .unwrap_or_else(|| infer_project_root_without_manifest(&first.file_path)); + let project_root = absolute_project_root(&project_root); + let source_root = common::resolve_source_root(&project_root, manifest.as_ref()); - let path_display = first.file_path.to_string_lossy(); - let mut ast = match parser::parse_with_context_and_surfaces( - &tokens, - Some(path_display.as_ref()), - Some(&library_imported_vocab), - Some(&library_imported_dsl_surfaces), + let inline_module_batch = match prepare_inline_source_module_batch( + &sources_by_file, + conftest_files_by_file, + &source_root, + &library_manifest_index, + &library_imported_vocab, + &library_imported_dsl_surfaces, ) { - Ok(a) => a, - Err(e) => { - return tests - .iter() - .map(|t| { - ( - t.clone(), - TestResult::Failed(start.elapsed(), format!("Parser error: {:?}", e)), - ) - }) - .collect(); - } - }; - if let Err(errors) = - vocab_desugar_pass::desugar_program_vocab_blocks(&mut ast, Some(path_display.as_ref()), &library_manifest_index) - { - return tests - .iter() - .map(|t| { - ( - t.clone(), - TestResult::Failed(start.elapsed(), format!("Vocab desugar error: {:?}", errors)), - ) - }) - .collect(); - } - let mut runner_ast = ast_with_inline_test_declarations(&ast); - normalize_runner_assert_statements(&mut runner_ast); - prune_shadowed_fixture_declarations(&mut runner_ast); - dedupe_import_declarations(&mut runner_ast); - let mut fixtures = collect_fixture_execution_info(&runner_ast, &HashMap::new()); - let fixture_teardowns = match split_yield_fixture_declarations(&mut runner_ast) { - Ok(teardowns) => teardowns, + Ok(batch) => batch, Err(message) => { return tests .iter() @@ -2497,7 +2827,78 @@ pub(super) fn run_file_tests_batch( .collect(); } }; - apply_fixture_teardowns(&mut fixtures, &fixture_teardowns); + + let (runner_ast, fixtures, source_modules, module_harnesses) = if let Some(batch) = inline_module_batch { + (batch.ast, HashMap::new(), batch.source_modules, batch.harnesses) + } else { + if batch_has_cross_file_top_level_collision(&sources_by_file, Some(&library_imported_vocab)) { + let mut split_results = Vec::new(); + for file_group in partition_collision_free_file_groups(&sources_by_file, Some(&library_imported_vocab)) { + let file_group = file_group.into_iter().collect::>(); + let file_tests = tests + .iter() + .filter(|test| file_group.contains(&test.file_path)) + .cloned() + .collect::>(); + split_results.extend(run_file_tests_batch( + &file_tests, + conftest_files_by_file, + prep_cache, + cargo_policy, + cargo_features, + cargo_no_default_features, + cargo_all_features, + options, + )); + } + return split_results; + } + + let ast = match parse_and_desugar_test_sources( + &batch_parse_sources, + &library_manifest_index, + &library_imported_vocab, + &library_imported_dsl_surfaces, + ) { + Ok(ast) => ast, + Err(message) => { + return tests + .iter() + .map(|t| (t.clone(), TestResult::Failed(start.elapsed(), message.clone()))) + .collect(); + } + }; + let (runner_ast, fixtures) = match prepare_runner_program(&ast) { + Ok(prepared) => prepared, + Err(message) => { + return tests + .iter() + .map(|t| (t.clone(), TestResult::Failed(start.elapsed(), message.clone()))) + .collect(); + } + }; + let source_modules = match collect_source_modules_for_test( + &runner_ast, + &source_root, + Some(&library_imported_vocab), + Some(&library_imported_dsl_surfaces), + Some(&library_manifest_index), + ) { + Ok(m) => m, + Err(e) => { + return tests + .iter() + .map(|t| { + ( + t.clone(), + TestResult::Failed(start.elapsed(), format!("Failed to collect source modules: {}", e)), + ) + }) + .collect(); + } + }; + (runner_ast, fixtures, source_modules, Vec::new()) + }; let cargo_feature_selection = CargoFeatureSelection { cargo_features: cargo_features.to_vec(), @@ -2506,34 +2907,6 @@ pub(super) fn run_file_tests_batch( } .normalized(); - // ---- Context: resolve project paths and collect transitive Incan modules for the test ---- - let project_root = manifest - .as_ref() - .map(|m| m.project_root().to_path_buf()) - .unwrap_or_else(|| infer_project_root_without_manifest(&first.file_path)); - let project_root = absolute_project_root(&project_root); - let source_root = common::resolve_source_root(&project_root, manifest.as_ref()); - let source_modules = match collect_source_modules_for_test( - &runner_ast, - &source_root, - Some(&library_imported_vocab), - Some(&library_imported_dsl_surfaces), - Some(&library_manifest_index), - ) { - Ok(m) => m, - Err(e) => { - return tests - .iter() - .map(|t| { - ( - t.clone(), - TestResult::Failed(start.elapsed(), format!("Failed to collect source modules: {}", e)), - ) - }) - .collect(); - } - }; - // ---- Context: session prep cache — reuse deps / lock / rust-inspect when key matches ---- let cache_key = compute_test_prep_cache_key( &first.file_path, @@ -2583,7 +2956,7 @@ pub(super) fn run_file_tests_batch( }; let mut resolved = - match resolve_dependencies(manifest.as_ref(), &inline_imports, true, &cargo_feature_selection) { + match resolve_reachable_dependencies(manifest.as_ref(), &inline_imports, true, &cargo_feature_selection) { Ok(resolved) => resolved, Err(errors) => { let mut sources = HashMap::new(); @@ -2634,21 +3007,25 @@ pub(super) fn run_file_tests_batch( .collect(); } }; - lock_resolved = - match resolve_dependencies(manifest.as_ref(), &lock_inline_imports, true, &cargo_feature_selection) { - Ok(resolved) => resolved, - Err(errors) => { - let sources = common::build_source_map(&lock_dependency_modules); - let mut msg = String::new(); - for err in &errors { - msg.push_str(&common::format_dependency_error(err, &sources)); - } - return tests - .iter() - .map(|t| (t.clone(), TestResult::Failed(start.elapsed(), msg.clone()))) - .collect(); + lock_resolved = match resolve_reachable_dependencies( + manifest.as_ref(), + &lock_inline_imports, + true, + &cargo_feature_selection, + ) { + Ok(resolved) => resolved, + Err(errors) => { + let sources = common::build_source_map(&lock_dependency_modules); + let mut msg = String::new(); + for err in &errors { + msg.push_str(&common::format_dependency_error(err, &sources)); } - }; + return tests + .iter() + .map(|t| (t.clone(), TestResult::Failed(start.elapsed(), msg.clone()))) + .collect(); + } + }; if let Err(err) = common::merge_project_requirement_dependencies(&mut lock_resolved, &lock_project_requirements) { @@ -2745,6 +3122,7 @@ pub(super) fn run_file_tests_batch( library_manifest_index, ast: runner_ast, fixtures, + module_harnesses, source_modules, project_root, resolved: cargo_resolved, @@ -2772,7 +3150,26 @@ pub(super) fn run_file_tests_batch( codegen.add_module_with_path_segments(&module.name, &module.ast, module.path_segments.clone()); } let fixtures = prepared.fixtures.clone(); - codegen.set_externally_reachable_items(collect_harness_entrypoints(tests, &fixtures)); + if prepared.module_harnesses.is_empty() { + codegen.set_externally_reachable_items(collect_harness_entrypoints(tests, &fixtures)); + } else { + let reachable_by_module = prepared + .module_harnesses + .iter() + .map(|harness| { + let file_tests = tests + .iter() + .filter(|test| test.file_path == harness.file_path) + .cloned() + .collect::>(); + ( + harness.module_path.clone(), + collect_harness_entrypoints(&file_tests, &harness.fixtures), + ) + }) + .collect::>(); + codegen.set_externally_reachable_items_by_module(reachable_by_module); + } let batch_file_paths = tests.iter().map(|test| test.file_path.clone()).collect::>(); let dir_suffix = file_batch_dir_suffix(&batch_file_paths); @@ -2789,10 +3186,11 @@ pub(super) fn run_file_tests_batch( let mut generator = ProjectGenerator::new(&temp_dir, &runner_crate_name, false); generator.set_package_name(Some(prepared.project_name.clone())); - generator.set_stdlib_features(test_runner_stdlib_features( + generator.set_stdlib_features(test_runner_stdlib_features_for_batch( &prepared.project_requirements.stdlib_features, tests, &fixtures, + &prepared.module_harnesses, )); generator.set_cargo_lock_payload(prepared.lock_payload.clone()); let cargo_flags = common::cargo_command_flags(cargo_policy, &cargo_feature_selection); @@ -2830,11 +3228,40 @@ pub(super) fn run_file_tests_batch( .iter() .map(|m| m.path_segments.clone()) .collect(); - let (main_code, rust_modules) = match codegen.try_generate_multi_file_nested(&prepared.ast, &module_paths) { - Ok(result) => result, - Err(e) => return gen_err(format!("Code generation error: {}", e)), - }; - let main_code = inject_file_test_harness(&main_code, tests, &prepared.project_root, &fixtures); + let (mut main_code, mut rust_modules) = + match codegen.try_generate_multi_file_nested(&prepared.ast, &module_paths) { + Ok(result) => result, + Err(e) => return gen_err(format!("Code generation error: {}", e)), + }; + if prepared.module_harnesses.is_empty() { + main_code = inject_file_test_harness(&main_code, tests, &prepared.project_root, &fixtures); + } else { + for harness in &prepared.module_harnesses { + let tests_with_indices = tests + .iter() + .enumerate() + .filter(|(_, test)| test.file_path == harness.file_path) + .collect::>(); + let file_tests = tests_with_indices + .iter() + .map(|(_, test)| (*test).clone()) + .collect::>(); + let test_indices = tests_with_indices.iter().map(|(index, _)| *index).collect::>(); + let Some(module_code) = rust_modules.get_mut(&harness.module_path) else { + return gen_err(format!( + "generated test harness module `{}` was not emitted", + harness.module_path.join(".") + )); + }; + *module_code = inject_file_test_harness_with_indices( + module_code, + &file_tests, + &test_indices, + &prepared.project_root, + &harness.fixtures, + ); + } + } match generator.generate_nested(&main_code, &rust_modules) { Ok(changed) => changed, Err(e) => return gen_err(format!("Failed to generate project: {}", e)), @@ -3465,6 +3892,52 @@ test test_runner_76001490ba86f677::__incan_file_tests::incan_harness_1_b ... FAI assert_eq!(name, "test_runner_76001490ba86f677"); } + #[test] + fn partition_collision_free_file_groups_considers_import_bindings() { + let sources = vec![ + ( + PathBuf::from("tests/test_imports_col.incn"), + "from helpers import col\n\ndef test_imported_col() -> None:\n assert col() == 1\n".to_string(), + ), + ( + PathBuf::from("tests/test_declares_col.incn"), + "def col() -> int:\n return 2\n\ndef test_local_col() -> None:\n assert col() == 2\n".to_string(), + ), + ]; + + let groups = partition_collision_free_file_groups(&sources, None); + + assert_eq!(groups.len(), 2); + } + + #[test] + fn partition_collision_free_file_groups_allows_repeated_import_bindings() { + let sources = vec![ + ( + PathBuf::from("tests/test_a.incn"), + "from std.testing import assert_eq\n\ndef test_a() -> None:\n assert_eq(1, 1)\n".to_string(), + ), + ( + PathBuf::from("tests/test_b.incn"), + "from std.testing import assert_eq\n\ndef test_b() -> None:\n assert_eq(2, 2)\n".to_string(), + ), + ]; + + let groups = partition_collision_free_file_groups(&sources, None); + + assert_eq!(groups.len(), 1); + } + + #[test] + fn module_name_for_segments_disambiguates_join_collisions() { + let flat = module_name_for_segments(&["a_b".to_string()]); + let nested = module_name_for_segments(&["a".to_string(), "b".to_string()]); + + assert_ne!(flat, nested); + assert!(flat.starts_with("a_b_")); + assert!(nested.starts_with("a_b_")); + } + #[test] fn inject_file_test_harness_emits_tests_module() { let rust = "fn test_a() {}\nfn test_b() {}\n"; diff --git a/src/cli/test_runner/module_graph.rs b/src/cli/test_runner/module_graph.rs index 6f579c68d..9378911fe 100644 --- a/src/cli/test_runner/module_graph.rs +++ b/src/cli/test_runner/module_graph.rs @@ -3,7 +3,8 @@ use std::fs; use std::path::{Path, PathBuf}; use crate::cli::commands::common::{ - resolve_stdlib_module_source_path, uses_iterator_adapter_surface, uses_result_combinator_surface, + resolve_stdlib_module_source_path, topologically_sort_modules, uses_iterator_adapter_surface, + uses_result_combinator_surface, }; use crate::cli::prelude::ParsedModule; use crate::frontend::ast::Program; @@ -24,7 +25,7 @@ fn queue_incan_stdlib_source_module( incan_source_stdlib_module_paths: &mut HashMap, processed: &HashSet, to_process: &mut Vec<(PathBuf, String, Vec)>, -) -> Result<(), String> { +) -> Result, String> { let stdlib_key = module_path.join("."); let source_path = if let Some(cached_path) = incan_source_stdlib_module_paths.get(&stdlib_key) { cached_path.clone() @@ -37,9 +38,9 @@ fn queue_incan_stdlib_source_module( module_segments.extend(module_path.iter().skip(1).cloned()); let module_name = module_segments.join("_"); if !processed.contains(&source_path) { - to_process.push((source_path, module_name, module_segments)); + to_process.push((source_path.clone(), module_name, module_segments)); } - Ok(()) + Ok(Some(source_path)) } /// Queue one canonical source-import resolution for test dependency collection. @@ -48,26 +49,31 @@ fn queue_resolved_source_import( incan_source_stdlib_module_paths: &mut HashMap, processed: &HashSet, to_process: &mut Vec<(PathBuf, String, Vec)>, -) -> Result<(), String> { +) -> Result, String> { match resolution { SourceModuleImportResolution::Stdlib { module_path } => { if stdlib::stdlib_stub_path(&module_path).is_some() { - queue_incan_stdlib_source_module( + return queue_incan_stdlib_source_module( &module_path, incan_source_stdlib_module_paths, processed, to_process, - )?; + ); } } SourceModuleImportResolution::Local(module_ref) => { if !processed.contains(&module_ref.file_path) { - to_process.push((module_ref.file_path, module_ref.module_name, module_ref.path_segments)); + to_process.push(( + module_ref.file_path.clone(), + module_ref.module_name, + module_ref.path_segments, + )); } + return Ok(Some(module_ref.file_path)); } SourceModuleImportResolution::External => {} } - Ok(()) + Ok(None) } /// Queue implicit source stdlib helper modules that generated Rust may reference without a source import. @@ -76,9 +82,10 @@ fn queue_implicit_stdlib_helpers( incan_source_stdlib_module_paths: &mut HashMap, processed: &HashSet, to_process: &mut Vec<(PathBuf, String, Vec)>, -) -> Result<(), String> { - if uses_iterator_adapter_surface(program) { - queue_incan_stdlib_source_module( +) -> Result, String> { + let mut queued = Vec::new(); + if uses_iterator_adapter_surface(program) + && let Some(path) = queue_incan_stdlib_source_module( &[ stdlib::STDLIB_ROOT.to_string(), "derives".to_string(), @@ -87,17 +94,25 @@ fn queue_implicit_stdlib_helpers( incan_source_stdlib_module_paths, processed, to_process, - )?; + )? + { + queued.push(path); } - if uses_result_combinator_surface(program) { - queue_incan_stdlib_source_module( + if uses_result_combinator_surface(program) + && let Some(path) = queue_incan_stdlib_source_module( &[stdlib::STDLIB_ROOT.to_string(), "result".to_string()], incan_source_stdlib_module_paths, processed, to_process, - )?; + )? + { + queued.push(path); } - Ok(()) + Ok(queued) +} + +fn dependency_edge_key(path: &Path) -> String { + path.to_string_lossy().to_string() } /// Collect source modules referenced by a test file's imports. @@ -118,6 +133,7 @@ pub(crate) fn collect_source_modules_for_test( let mut processed = HashSet::new(); let mut to_process: Vec<(PathBuf, String, Vec)> = Vec::new(); let mut incan_source_stdlib_module_paths: HashMap = HashMap::new(); + let mut dependency_edges: HashMap> = HashMap::new(); queue_implicit_stdlib_helpers( test_ast, @@ -128,7 +144,7 @@ pub(crate) fn collect_source_modules_for_test( // ---- Walk test AST to find user module imports ---- for resolved in resolve_program_source_imports(test_ast, source_root, Some(source_root)) { - queue_resolved_source_import( + let _ = queue_resolved_source_import( resolved.resolution, &mut incan_source_stdlib_module_paths, &processed, @@ -142,6 +158,8 @@ pub(crate) fn collect_source_modules_for_test( continue; } processed.insert(file_path.clone()); + let file_key = dependency_edge_key(&file_path); + dependency_edges.entry(file_key.clone()).or_default(); let source = fs::read_to_string(&file_path) .map_err(|e| format!("Failed to read source module '{}': {}", file_path.display(), e))?; @@ -184,17 +202,29 @@ pub(crate) fn collect_source_modules_for_test( eprint!("{}", diagnostics::format_error(&fp, &source, warn)); } - queue_implicit_stdlib_helpers(&ast, &mut incan_source_stdlib_module_paths, &processed, &mut to_process)?; + for dependency_path in + queue_implicit_stdlib_helpers(&ast, &mut incan_source_stdlib_module_paths, &processed, &mut to_process)? + { + dependency_edges + .entry(file_key.clone()) + .or_default() + .insert(dependency_edge_key(&dependency_path)); + } // Walk this module's imports for transitive dependencies. let current_base = file_path.parent().unwrap_or(source_root); for resolved in resolve_program_source_imports(&ast, current_base, Some(source_root)) { - queue_resolved_source_import( + if let Some(dependency_path) = queue_resolved_source_import( resolved.resolution, &mut incan_source_stdlib_module_paths, &processed, &mut to_process, - )?; + )? { + dependency_edges + .entry(file_key.clone()) + .or_default() + .insert(dependency_edge_key(&dependency_path)); + } } modules.push(ParsedModule { @@ -206,7 +236,7 @@ pub(crate) fn collect_source_modules_for_test( }); } - Ok(modules) + topologically_sort_modules(modules, &dependency_edges).map_err(|err| err.message) } #[cfg(test)] @@ -257,6 +287,39 @@ mod tests { Ok(()) } + #[test] + fn test_runner_orders_source_dependencies_before_dependents() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let src_dir = tmp.path().join("src"); + std::fs::create_dir_all(&src_dir)?; + std::fs::write(src_dir.join("helper.incn"), "pub def target() -> int:\n return 1\n")?; + std::fs::write( + src_dir.join("functions.incn"), + "from helper import target as target_builder\n\npub public_target = alias target_builder\n", + )?; + + let test_source = "from functions import public_target\n"; + let tokens = lexer::lex(test_source).map_err(|errs| errs[0].message.clone())?; + let ast = parser::parse_with_context(&tokens, Some("tests/test_alias.incn"), None) + .map_err(|errs| errs[0].message.clone())?; + + let modules = collect_source_modules_for_test(&ast, &src_dir, None, None, None)?; + let helper_idx = modules + .iter() + .position(|module| module.file_path.ends_with("helper.incn")) + .ok_or("expected helper.incn to be collected")?; + let functions_idx = modules + .iter() + .position(|module| module.file_path.ends_with("functions.incn")) + .ok_or("expected functions.incn to be collected")?; + + assert!( + helper_idx < functions_idx, + "test runner should order dependency modules before dependent modules" + ); + Ok(()) + } + #[test] fn test_runner_collects_implicit_result_helper_modules() -> Result<(), Box> { let tmp = tempfile::tempdir()?; diff --git a/src/dependency_resolver.rs b/src/dependency_resolver.rs index 1392da575..2c44b6e04 100644 --- a/src/dependency_resolver.rs +++ b/src/dependency_resolver.rs @@ -15,7 +15,7 @@ use crate::frontend::ast::Span; use crate::frontend::diagnostics::CompileError; use crate::lockfile::CargoFeatureSelection; use crate::manifest::{DependencySource, DependencySpec, ProjectManifest}; -use incan_core::lang::stdlib::{self, STDLIB_NAMESPACES, StdlibExtraCrateSource}; +use incan_core::lang::stdlib::{self, StdlibExtraCrateSource}; /// Validate that a version requirement string uses Cargo SemVer syntax. /// @@ -54,6 +54,12 @@ pub struct ResolvedDependencies { pub dev_dependencies: Vec, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ManifestDependencyScope { + All, + ReachableOnly, +} + fn with_rust_import_context(error: CompileError, import: &InlineRustImport) -> CompileError { error .with_note(format!("import site: `{}`", import.import_path)) @@ -65,6 +71,37 @@ pub fn resolve_dependencies( inline_imports: &[InlineRustImport], include_dev_dependencies: bool, cargo_features: &CargoFeatureSelection, +) -> Result> { + resolve_dependencies_with_scope( + manifest, + inline_imports, + include_dev_dependencies, + cargo_features, + ManifestDependencyScope::All, + ) +} + +pub fn resolve_reachable_dependencies( + manifest: Option<&ProjectManifest>, + inline_imports: &[InlineRustImport], + include_dev_dependencies: bool, + cargo_features: &CargoFeatureSelection, +) -> Result> { + resolve_dependencies_with_scope( + manifest, + inline_imports, + include_dev_dependencies, + cargo_features, + ManifestDependencyScope::ReachableOnly, + ) +} + +fn resolve_dependencies_with_scope( + manifest: Option<&ProjectManifest>, + inline_imports: &[InlineRustImport], + include_dev_dependencies: bool, + cargo_features: &CargoFeatureSelection, + scope: ManifestDependencyScope, ) -> Result> { let mut errors = Vec::new(); @@ -100,14 +137,24 @@ pub fn resolve_dependencies( ); // Combine manifest deps with resolved inline specs. - let mut resolved_deps: HashMap = manifest_deps.clone(); + let mut resolved_deps: HashMap = match scope { + ManifestDependencyScope::All => manifest_deps.clone(), + ManifestDependencyScope::ReachableOnly => { + select_manifest_dependencies(&manifest_deps, &inline_merge.manifest_dependency_keys) + } + }; let mut resolved_dev_deps: HashMap = if include_dev_dependencies { - manifest_dev_deps.clone() + match scope { + ManifestDependencyScope::All => manifest_dev_deps.clone(), + ManifestDependencyScope::ReachableOnly => { + select_manifest_dependencies(&manifest_dev_deps, &inline_merge.manifest_dev_dependency_keys) + } + } } else { HashMap::new() }; - for (crate_name, inline) in inline_merge { + for (crate_name, inline) in inline_merge.inline_specs { if inline.is_test_only { if include_dev_dependencies { resolved_dev_deps.insert(crate_name, inline.spec); @@ -141,6 +188,13 @@ pub fn resolve_dependencies( // Inline merge + validation // ============================================================================ +#[derive(Default)] +struct InlineMergeResult { + inline_specs: HashMap, + manifest_dependency_keys: HashSet, + manifest_dev_dependency_keys: HashSet, +} + struct InlineMergedSpec { spec: DependencySpec, is_test_only: bool, @@ -162,8 +216,10 @@ fn merge_inline_imports( manifest_dev_deps: &HashMap, library_dep_names: &HashSet, errors: &mut Vec, -) -> HashMap { +) -> InlineMergeResult { let mut merged: HashMap = HashMap::new(); + let mut manifest_dependency_keys = HashSet::new(); + let mut manifest_dev_dependency_keys = HashSet::new(); for import in inline_imports { if import.crate_name == stdlib::STDLIB_ROOT { @@ -229,6 +285,13 @@ fn merge_inline_imports( continue; } + if let Some((key, _)) = manifest_dep_match { + manifest_dependency_keys.insert(key.clone()); + } + if let Some((key, _)) = manifest_dev_dep_match { + manifest_dev_dependency_keys.insert(key.clone()); + } + if manifest_dep_match.is_some() || manifest_dev_dep_match.is_some() { if has_inline_spec { errors.push(DependencyError { @@ -306,21 +369,6 @@ fn merge_inline_imports( let mut resolved = HashMap::new(); for (crate_name, mut merged_spec) in merged { if merged_spec.spec.version.is_none() { - if !merged_spec.spec.features.is_empty() { - errors.push(DependencyError { - file_path: merged_spec.first_site.file_path.clone(), - error: with_rust_import_context( - CompileError::new( - format!("Rust import features for `{}` require a version annotation", crate_name), - merged_spec.first_site.span, - ) - .with_hint("Add `@ \"version\"` to the rust import."), - &merged_spec.first_site, - ), - }); - continue; - } - let Some(default) = known_good_spec(&crate_name) else { errors.push(DependencyError { file_path: merged_spec.first_site.file_path.clone(), @@ -337,13 +385,34 @@ fn merge_inline_imports( }); continue; }; + let requested_features = std::mem::take(&mut merged_spec.spec.features); merged_spec.spec = default; + for feature in requested_features { + if !merged_spec.spec.features.contains(&feature) { + merged_spec.spec.features.push(feature); + } + } + merged_spec.spec = merged_spec.spec.normalized(); } resolved.insert(crate_name, merged_spec); } - resolved + InlineMergeResult { + inline_specs: resolved, + manifest_dependency_keys, + manifest_dev_dependency_keys, + } +} + +fn select_manifest_dependencies( + deps: &HashMap, + selected_keys: &HashSet, +) -> HashMap { + deps.iter() + .filter(|(key, _)| selected_keys.contains(*key)) + .map(|(key, spec)| (key.clone(), spec.clone())) + .collect() } /// Convert one inline `rust::` import annotation into the dependency spec emitted to generated Cargo manifests. @@ -355,20 +424,11 @@ fn inline_spec_from_import(import: &InlineRustImport) -> DependencySpec { default_features: true, source: DependencySource::Registry, optional: false, - package: rust_crate_package_alias(&import.crate_name).map(str::to_string), + package: stdlib::extra_crate_package_alias(&import.crate_name).map(str::to_string), } .normalized() } -/// Return the published Cargo package name when it differs from the Rust crate import path. -fn rust_crate_package_alias(crate_name: &str) -> Option<&'static str> { - match crate_name { - "md5" => Some("md-5"), - "xxhash_rust" => Some("xxhash-rust"), - _ => None, - } -} - fn merge_inline_spec(existing: &mut InlineMergedSpec, next: &InlineRustImport) -> Result<(), String> { let next_version = next.version.clone(); if existing.spec.version != next_version { @@ -500,6 +560,10 @@ fn validate_optional_imports( // ============================================================================ fn known_good_spec(crate_name: &str) -> Option { + if let Some(spec) = known_good_spec_from_stdlib(crate_name) { + return Some(spec); + } + let (version, features): (&str, Vec<&str>) = match crate_name { "serde" => ("1.0", vec!["derive"]), "serde_json" => ("1.0", vec![]), @@ -508,8 +572,6 @@ fn known_good_spec(crate_name: &str) -> Option { "chrono" => ("0.4", vec!["serde"]), "reqwest" => ("0.11", vec!["json"]), "uuid" => ("1.0", vec!["v4", "serde"]), - "rand" => ("0.8", vec![]), - "regex" => ("1.0", vec![]), "anyhow" => ("1.0", vec![]), "thiserror" => ("1.0", vec![]), "tracing" => ("0.1", vec![]), @@ -520,10 +582,7 @@ fn known_good_spec(crate_name: &str) -> Option { "futures" => ("0.3", vec![]), "bytes" => ("1.0", vec![]), "itertools" => ("0.12", vec![]), - // For any crate not in the hardcoded list above, fall through to the stdlib registry. - // STDLIB_NAMESPACES is the single source of truth for stdlib-managed crate versions, - // so we derive the spec from there rather than duplicating version strings here. - _ => return known_good_spec_from_stdlib(crate_name), + _ => return None, }; Some( @@ -542,33 +601,27 @@ fn known_good_spec(crate_name: &str) -> Option { /// Look up a known-good spec for crates declared as `extra_crate_deps` in any stdlib namespace. /// -/// This makes `STDLIB_NAMESPACES` the single source of truth for stdlib-managed crate versions. -/// When a stdlib `.incn` file writes `from rust::axum import ...` without an inline version annotation, the resolver -/// finds the version here rather than requiring a duplicate hardcoded entry in `known_good_spec`. +/// This makes the stdlib registry the single source of truth for stdlib-managed crate versions. When a stdlib `.incn` +/// file writes `from rust::axum import ...` without an inline version annotation, the resolver finds the version here +/// rather than requiring a duplicate hardcoded entry in `known_good_spec`. fn known_good_spec_from_stdlib(crate_name: &str) -> Option { - for ns in STDLIB_NAMESPACES { - for dep in ns.extra_crate_deps { - if dep.crate_name == crate_name { - let StdlibExtraCrateSource::Version(version) = dep.source else { - // Path dependencies are not registry crates; skip. - continue; - }; - return Some( - DependencySpec { - crate_name: crate_name.to_string(), - version: Some(version.to_string()), - features: vec![], - default_features: true, - source: DependencySource::Registry, - optional: false, - package: None, - } - .normalized(), - ); - } + let dep = stdlib::extra_crate_deps() + .find(|dep| dep.crate_name == crate_name && matches!(dep.source, StdlibExtraCrateSource::Version(_)))?; + let StdlibExtraCrateSource::Version(version) = dep.source else { + return None; + }; + Some( + DependencySpec { + crate_name: crate_name.to_string(), + version: Some(version.to_string()), + features: dep.features.iter().map(|feature| (*feature).to_string()).collect(), + default_features: true, + source: DependencySource::Registry, + optional: false, + package: stdlib::extra_crate_package_alias(crate_name).map(str::to_string), } - } - None + .normalized(), + ) } #[cfg(test)] @@ -614,6 +667,16 @@ mod tests { .map_err(|errors| std::io::Error::other(format!("{errors:?}")).into()) } + fn resolve_reachable_ok( + manifest: Option<&ProjectManifest>, + inline_imports: &[InlineRustImport], + include_dev_dependencies: bool, + cargo_features: &CargoFeatureSelection, + ) -> TestResult { + resolve_reachable_dependencies(manifest, inline_imports, include_dev_dependencies, cargo_features) + .map_err(|errors| std::io::Error::other(format!("{errors:?}")).into()) + } + fn dependency<'a>(deps: &'a [DependencySpec], crate_name: &str) -> TestResult<&'a DependencySpec> { deps.iter() .find(|dep| dep.crate_name == crate_name) @@ -734,6 +797,41 @@ serde = "1.0" Ok(()) } + #[test] + fn reachable_resolution_omits_unused_manifest_dependency() -> TestResult { + let toml_str = r#" +[rust-dependencies] +datafusion = "53" +"#; + let manifest = parse_manifest(toml_str)?; + + let resolved = resolve_reachable_ok(Some(&manifest), &[], false, &default_cargo_features())?; + + assert!( + !resolved + .dependencies + .iter() + .any(|dependency| dependency.crate_name == "datafusion"), + "reachable resolution should not emit unused manifest dependencies: {resolved:?}" + ); + Ok(()) + } + + #[test] + fn reachable_resolution_keeps_imported_manifest_dependency() -> TestResult { + let toml_str = r#" +[rust-dependencies] +serde = "1.0" +"#; + let manifest = parse_manifest(toml_str)?; + let imports = vec![inline("serde", None, &[], false)]; + + let resolved = resolve_reachable_ok(Some(&manifest), &imports, false, &default_cargo_features())?; + let serde = dependency(&resolved.dependencies, "serde")?; + assert_eq!(serde.version.as_deref(), Some("1.0")); + Ok(()) + } + // ---- Phase 3: Dev-dep gating (test context only) ---- #[test] @@ -794,6 +892,51 @@ test_lib = "0.5" Ok(()) } + #[test] + fn known_good_default_allows_features_without_inline_version() -> TestResult { + let imports = vec![inline("tokio", None, &["full"], false)]; + + let resolved = resolve_ok(None, &imports, false, &default_cargo_features())?; + let tokio = dependency(&resolved.dependencies, "tokio")?; + assert_eq!(tokio.version.as_deref(), Some("1")); + assert!(tokio.features.contains(&"rt-multi-thread".to_string())); + assert!(tokio.features.contains(&"full".to_string())); + Ok(()) + } + + #[test] + fn stdlib_registry_version_dependencies_drive_known_good_defaults() -> TestResult { + for ns in stdlib::STDLIB_NAMESPACES { + for dep in ns.extra_crate_deps { + let StdlibExtraCrateSource::Version(version) = dep.source else { + continue; + }; + let spec = known_good_spec(dep.crate_name).ok_or_else(|| { + std::io::Error::other(format!( + "expected registry dependency `{}` to resolve as a known-good default", + dep.crate_name + )) + })?; + assert_eq!( + spec.version.as_deref(), + Some(version), + "dependency resolver drifted from stdlib registry metadata for `{}`", + dep.crate_name + ); + assert_eq!( + spec.features, + dep.features + .iter() + .map(|feature| (*feature).to_string()) + .collect::>(), + "dependency resolver drifted from stdlib registry feature metadata for `{}`", + dep.crate_name + ); + } + } + Ok(()) + } + #[test] fn unknown_crate_without_version_is_error() -> TestResult { let imports = vec![inline("unknown_crate_xyz", None, &[], false)]; diff --git a/src/format/formatter/declarations.rs b/src/format/formatter/declarations.rs index 9e56fe3d7..f0f05b2ba 100644 --- a/src/format/formatter/declarations.rs +++ b/src/format/formatter/declarations.rs @@ -1048,6 +1048,16 @@ impl Formatter { fn format_decorator(&mut self, dec: &Decorator) { self.writer.write("@"); self.format_decorator_path(&dec.path); + if !dec.type_args.is_empty() { + self.writer.write("["); + for (idx, arg) in dec.type_args.iter().enumerate() { + if idx > 0 { + self.writer.write(", "); + } + self.format_type(&arg.node); + } + self.writer.write("]"); + } if dec.is_call { self.writer.write("("); for (i, arg) in dec.args.iter().enumerate() { diff --git a/src/format/formatter/expressions.rs b/src/format/formatter/expressions.rs index b0e672c6a..23ebc7bb1 100644 --- a/src/format/formatter/expressions.rs +++ b/src/format/formatter/expressions.rs @@ -289,7 +289,7 @@ impl Formatter { } Expr::Closure(params, body) => { self.writer.write("("); - self.format_params(params); + self.format_closure_params(params); self.writer.write(") => "); self.format_expr(&body.node); } @@ -372,9 +372,12 @@ impl Formatter { for part in parts { match part { FStringPart::Literal(s) => self.writer.write(&escape_fstring_literal(s)), - FStringPart::Expr(expr) => { + FStringPart::Expr { expr, format } => { self.writer.write("{"); self.format_expr(&expr.node); + if matches!(format, FStringFormat::Debug) { + self.writer.write(":?"); + } self.writer.write("}"); } } @@ -452,7 +455,7 @@ impl Formatter { match clause { ComprehensionClause::For { pattern, iter } => { self.writer.write(" for "); - self.format_pattern(&pattern.node); + self.format_for_pattern(&pattern.node); self.writer.write(" in "); self.format_expr(&iter.node); } @@ -540,6 +543,15 @@ impl Formatter { // ---- Call args ---- + fn format_closure_params(&mut self, params: &[Spanned]) { + for (i, param) in params.iter().enumerate() { + if i > 0 { + self.writer.write(", "); + } + self.writer.write(¶m.node.name); + } + } + fn format_call_args(&mut self, args: &[CallArg]) { for (i, arg) in args.iter().enumerate() { if i > 0 { diff --git a/src/format/formatter/statements.rs b/src/format/formatter/statements.rs index 62bff7489..50e0bff5d 100644 --- a/src/format/formatter/statements.rs +++ b/src/format/formatter/statements.rs @@ -285,7 +285,8 @@ impl Formatter { self.writer.dedent(); } - fn format_for_pattern(&mut self, pattern: &Pattern) { + /// Format a `for`-target pattern using the grammar's unparenthesized tuple-target spelling. + pub(super) fn format_for_pattern(&mut self, pattern: &Pattern) { if let Pattern::Tuple(items) = pattern { for (i, item) in items.iter().enumerate() { if i > 0 { diff --git a/src/format/mod.rs b/src/format/mod.rs index b707b0fdd..4d3c89d9d 100644 --- a/src/format/mod.rs +++ b/src/format/mod.rs @@ -367,6 +367,20 @@ mod tests { Ok(()) } + #[test] + fn test_format_source_list_comprehension_tuple_target_omits_parentheses() -> Result<(), FormatError> { + let source = r#"def labels(values: list[str]) -> list[str]: + return [f"{idx}:{value}" for idx, value in enumerate(values)] +"#; + let expected = r#"def labels(values: list[str]) -> list[str]: + return [f"{idx}:{value}" for idx, value in enumerate(values)] +"#; + let formatted = format_source(source)?; + assert_eq!(formatted, expected); + assert_eq!(format_source(&formatted)?, expected); + Ok(()) + } + #[test] fn test_format_source_rfc028_operator_spellings() -> Result<(), FormatError> { let source = r#"def ops(a: Any, b: Any, c: Any) -> None: @@ -436,6 +450,34 @@ def MixedName() -> int: Ok(()) } + #[test] + fn test_format_source_decorator_factory_type_args() -> Result<(), FormatError> { + let source = r#"@registered[(str)->ColumnExpr]("inql.functions.col") +def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +"#; + let formatted = format_source(source)?; + let expected = r#"@registered[(str) -> ColumnExpr]("inql.functions.col") +def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +"#; + assert_eq!(formatted, expected); + Ok(()) + } + + #[test] + fn test_format_source_preserves_untyped_closure_params() -> Result<(), FormatError> { + let source = r#"pub def registered[F](_function_ref: str) -> (F) -> F: + return (func) => func +"#; + let formatted = format_source(source)?; + let expected = r#"pub def registered[F](_function_ref: str) -> (F) -> F: + return (func) => func +"#; + assert_eq!(formatted, expected); + Ok(()) + } + #[test] fn test_format_source_wraps_long_function_signature() -> Result<(), FormatError> { let source = r#"def append_node(store_id: int, kind: PrismNodeKind, input_ids: list[int], named_table: str, predicate: bool, limit_count: int) -> int: @@ -1091,6 +1133,18 @@ async def run() -> int: Ok(()) } + /// Regression (GitHub #625): f-string debug markers are semantic and must survive formatting. + #[test] + fn test_format_source_preserves_fstring_debug_marker() -> Result<(), FormatError> { + let source = "def main(columns: list[str]) -> str:\n return f\"columns: {columns:?}\"\n"; + let formatted = assert_format_round_trip_lex_parse(source)?; + assert!( + formatted.contains(r#"f"columns: {columns:?}""#), + "expected formatter to preserve f-string debug marker, got: {formatted}" + ); + Ok(()) + } + /// Regression #235: qualified constructor patterns use `::` in the AST; the formatter must print Incan surface `.`. #[test] fn test_format_source_qualified_match_pattern_round_trip() -> Result<(), FormatError> { diff --git a/src/frontend/api_metadata.rs b/src/frontend/api_metadata.rs index a9816f43f..608c53fb2 100644 --- a/src/frontend/api_metadata.rs +++ b/src/frontend/api_metadata.rs @@ -9,9 +9,9 @@ use std::collections::{HashMap, HashSet, VecDeque}; use serde::{Deserialize, Serialize}; use crate::frontend::ast::{ - ClassDecl, Declaration, Decorator, DecoratorArg, DecoratorArgValue, EnumDecl, Expr, FieldDecl, FunctionDecl, - ImportDecl, ImportItem, ImportKind, MethodDecl, ModelDecl, NewtypeDecl, Program, Span, Spanned, Statement, - TraitDecl, TypeAliasDecl, Visibility, + CallArg, ClassDecl, Declaration, Decorator, DecoratorArg, DecoratorArgValue, DictEntry, EnumDecl, Expr, FieldDecl, + FunctionDecl, ImportDecl, ImportItem, ImportKind, ListEntry, MethodDecl, ModelDecl, NewtypeDecl, Program, Span, + Spanned, Statement, TraitDecl, TypeAliasDecl, Visibility, }; use crate::frontend::decorator_resolution; use crate::frontend::diagnostics::CompileError; @@ -21,6 +21,7 @@ use crate::frontend::library_exports::{ CheckedPartialTargetKind, CheckedPresetValue, CheckedTraitExport, CheckedTypeAliasExport, CheckedTypeBound, CheckedTypeParam, collect_checked_public_exports, }; +use crate::frontend::module::canonicalize_source_module_segments; use crate::frontend::typechecker::{ConstValue, TypeChecker}; use crate::library_manifest::{ EnumValueExport, EnumValueTypeExport, FieldExport, ParamExport, ParamKindExport, PartialPresetExport, @@ -208,6 +209,8 @@ pub struct ApiAlias { pub name: String, pub anchor: SourceAnchor, pub target_path: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub projected_function: Option, } #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -244,7 +247,30 @@ pub struct DecoratorMetadata { pub path: Vec, pub source_name: String, pub anchor: SourceSpan, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub type_args: Vec, pub args: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub decorated_callable: Option, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct ApiProjectedFunction { + pub source_path: Vec, + pub callable: ApiCallableMetadata, + pub decorators: Vec, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct ApiCallableMetadata { + pub name: String, + pub anchor: SourceAnchor, + pub type_params: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub receiver: Option, + pub params: Vec, + pub return_type: TypeRef, + pub is_async: bool, } #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -264,6 +290,20 @@ pub enum DecoratorValue { name: String, value: Option, }, + SymbolRef { + path: Vec, + }, + List { + items: Vec, + }, + Dict { + entries: Vec, + }, + Call { + callee: Vec, + type_args: Vec, + args: Vec, + }, Type { ty: TypeRef, }, @@ -272,6 +312,21 @@ pub enum DecoratorValue { }, } +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct DecoratorDictEntry { + pub key: DecoratorValue, + pub value: DecoratorValue, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum DecoratorCallArgMetadata { + Positional { value: DecoratorValue }, + Named { name: String, value: DecoratorValue }, + PositionalUnpack { value: DecoratorValue }, + KeywordUnpack { value: DecoratorValue }, +} + #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(tag = "kind", content = "value", rename_all = "snake_case")] pub enum SafeMetadataValue { @@ -443,6 +498,7 @@ pub fn collect_checked_api_metadata( name: alias.name.clone(), anchor: anchor(&module_path, &alias.name, decl.span), target_path: alias.target.segments.clone(), + projected_function: None, })); } Declaration::Partial(partial) if public(partial.visibility) => { @@ -468,6 +524,106 @@ pub fn collect_checked_api_metadata( } } +/// Attach checked function projections to public aliases that target decorated or ordinary public functions. +/// +/// Metadata package consumers should not need to force producer module initialization just to discover declaration-side +/// decorator facts. This projection pass resolves aliases across the already checked API package and carries the target +/// function's decorators and checked callable shape onto facade aliases. +pub fn materialize_api_alias_projections(modules: &mut [CheckedApiMetadata]) { + let mut projections = HashMap::new(); + let mut aliases = Vec::new(); + + for module in modules.iter() { + for declaration in &module.declarations { + match declaration { + ApiDeclaration::Function(function) => { + projections.insert( + declaration_path(&module.module_path, &function.name), + ApiProjectedFunction { + source_path: declaration_path(&module.module_path, &function.name), + callable: callable_from_function(function), + decorators: function.decorators.clone(), + }, + ); + } + ApiDeclaration::Alias(alias) => aliases.push(ApiAliasProjectionRequest { + path: declaration_path(&module.module_path, &alias.name), + target_path: normalized_api_target_path(&alias.target_path), + name: alias.name.clone(), + anchor: alias.anchor.clone(), + }), + _ => {} + } + } + } + + let mut changed = true; + while changed { + changed = false; + for alias in &aliases { + if projections.contains_key(&alias.path) { + continue; + } + if let Some(target) = projections.get(&alias.target_path) { + projections.insert(alias.path.clone(), projected_function_for_alias(alias, target)); + changed = true; + } + } + } + + for module in modules { + for declaration in &mut module.declarations { + if let ApiDeclaration::Alias(alias) = declaration { + let alias_path = declaration_path(&module.module_path, &alias.name); + alias.projected_function = projections.get(&alias_path).cloned(); + } + } + } +} + +#[derive(Debug)] +struct ApiAliasProjectionRequest { + path: Vec, + target_path: Vec, + name: String, + anchor: SourceAnchor, +} + +fn declaration_path(module_path: &[String], name: &str) -> Vec { + let mut path = module_path.to_vec(); + path.push(name.to_string()); + path +} + +fn normalized_api_target_path(path: &[String]) -> Vec { + if path.first().is_some_and(|segment| segment == "crate") { + return path[1..].to_vec(); + } + path.to_vec() +} + +fn callable_from_function(function: &ApiFunction) -> ApiCallableMetadata { + ApiCallableMetadata { + name: function.name.clone(), + anchor: function.anchor.clone(), + type_params: function.type_params.clone(), + receiver: None, + params: function.params.clone(), + return_type: function.return_type.clone(), + is_async: function.is_async, + } +} + +fn projected_function_for_alias( + alias: &ApiAliasProjectionRequest, + target: &ApiProjectedFunction, +) -> ApiProjectedFunction { + let mut projected = target.clone(); + projected.callable.name = alias.name.clone(); + projected.callable.anchor = alias.anchor.clone(); + projected +} + fn checked_kind<'a>(exports: &'a HashMap, name: &str) -> Option<&'a CheckedExportKind> { exports.get(name).map(|export| &export.kind) } @@ -553,19 +709,70 @@ fn api_function( module_path: &[String], ) -> ApiFunction { let docstring = function_docstring(&function.body); + let callable = api_callable_for_function(function, span, export, checker, module_path); ApiFunction { - name: export.name.clone(), - anchor: anchor(module_path, &export.name, span), + name: callable.name.clone(), + anchor: callable.anchor.clone(), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&function.decorators, checker), + decorators: decorators_metadata(&function.decorators, checker, Some(&callable)), + type_params: callable.type_params, + params: callable.params, + return_type: callable.return_type, + is_async: callable.is_async, + } +} + +fn api_callable_for_function( + function: &FunctionDecl, + span: Span, + export: &CheckedFunctionExport, + checker: &TypeChecker, + module_path: &[String], +) -> ApiCallableMetadata { + ApiCallableMetadata { + name: export.name.clone(), + anchor: anchor(module_path, &export.name, span), type_params: type_params(&export.type_params), - params: params(&export.params), - return_type: type_ref_from_resolved(&export.return_type), - is_async: export.is_async, + receiver: None, + params: source_function_params(function, checker), + return_type: source_function_return_type(function, checker), + is_async: function.is_async(), } } +/// Build the source-declared callable parameter surface for API documentation metadata. +/// +/// User-defined decorators can rebind a public function symbol to an ordinary callable value. That callable type is the +/// right contract for lowering and invocation, but function API docs are attached to the source declaration and should +/// validate against the declaration's named parameters instead of an anonymous function-type projection. +fn source_function_params(function: &FunctionDecl, checker: &TypeChecker) -> Vec { + function + .params + .iter() + .map(|param| ParamExport { + name: param.node.name.clone(), + ty: type_ref_from_resolved(&crate::frontend::symbols::resolve_type( + ¶m.node.ty.node, + &checker.symbols, + )), + kind: match param.node.kind { + crate::frontend::ast::ParamKind::Normal => ParamKindExport::Normal, + crate::frontend::ast::ParamKind::RestPositional => ParamKindExport::RestPositional, + crate::frontend::ast::ParamKind::RestKeyword => ParamKindExport::RestKeyword, + }, + has_default: param.node.default.is_some(), + }) + .collect() +} + +fn source_function_return_type(function: &FunctionDecl, checker: &TypeChecker) -> TypeRef { + type_ref_from_resolved(&crate::frontend::symbols::resolve_type( + &function.return_type.node, + &checker.symbols, + )) +} + fn api_model( model: &ModelDecl, span: Span, @@ -579,7 +786,7 @@ fn api_model( anchor: anchor(module_path, &export.name, span), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&model.decorators, checker), + decorators: decorators_metadata(&model.decorators, checker, None), type_params: type_params(&export.type_params), traits: export.traits.clone(), derives: export.derives.clone(), @@ -601,7 +808,7 @@ fn api_class( anchor: anchor(module_path, &export.name, span), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&class.decorators, checker), + decorators: decorators_metadata(&class.decorators, checker, None), type_params: type_params(&export.type_params), extends: export.extends.clone(), traits: export.traits.clone(), @@ -625,7 +832,7 @@ fn api_trait( anchor: anchor(module_path, &export.name, span), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&trait_decl.decorators, checker), + decorators: decorators_metadata(&trait_decl.decorators, checker, None), type_params: type_params(&export.type_params), supertraits: export.supertraits.iter().map(type_bound).collect(), requires: export @@ -657,7 +864,7 @@ fn api_enum( anchor: anchor(module_path, &export.name, span), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&enum_decl.decorators, checker), + decorators: decorators_metadata(&enum_decl.decorators, checker, None), type_params: type_params(&export.type_params), value_type: export.value_type.map(|value_type| match value_type { crate::frontend::symbols::ValueEnumBacking::Str => EnumValueTypeExport::Str, @@ -700,7 +907,7 @@ fn api_newtype( anchor: anchor(module_path, &export.name, span), docstring_sections: parse_docstring(docstring.as_deref()), docstring, - decorators: decorators_metadata(&newtype.decorators, checker), + decorators: decorators_metadata(&newtype.decorators, checker, None), type_params: type_params(&export.type_params), is_rusttype: export.is_rusttype, underlying: type_ref_from_resolved(&export.underlying), @@ -743,7 +950,8 @@ fn api_const( fn api_aliases(import: &ImportDecl, span: Span, module_path: &[String]) -> Vec { match &import.kind { ImportKind::From { module, items } => { - let base_path = decorator_resolution::path_segments_with_prefix(module); + let base_path = + canonicalize_source_module_segments(&decorator_resolution::path_segments_with_prefix(module)); aliases_from_items(items, base_path, span, module_path) } ImportKind::RustFrom { @@ -780,6 +988,7 @@ fn aliases_from_items( anchor: anchor(module_path, &name, span), name, target_path, + projected_function: None, } }) .collect() @@ -809,12 +1018,9 @@ fn methods( continue; }; let docstring = method.node.body.as_ref().and_then(|body| function_docstring(body)); - out.push(ApiMethod { + let callable = ApiCallableMetadata { name: checked.name.clone(), anchor: anchor(module_path, &format!("{owner}.{}", checked.name), method.span), - docstring_sections: parse_docstring(docstring.as_deref()), - docstring, - decorators: decorators_metadata(&method.node.decorators, checker), type_params: type_params(&checked.type_params), receiver: checked.receiver.map(|receiver| match receiver { crate::frontend::ast::Receiver::Immutable => ReceiverExport::Immutable, @@ -823,6 +1029,18 @@ fn methods( params: params(&checked.params), return_type: type_ref_from_resolved(&checked.return_type), is_async: checked.is_async, + }; + out.push(ApiMethod { + name: callable.name.clone(), + anchor: callable.anchor.clone(), + docstring_sections: parse_docstring(docstring.as_deref()), + docstring, + decorators: decorators_metadata(&method.node.decorators, checker, Some(&callable)), + type_params: callable.type_params, + receiver: callable.receiver, + params: callable.params, + return_type: callable.return_type, + is_async: callable.is_async, has_body: checked.has_body, }); } @@ -969,7 +1187,11 @@ fn fields_in_source_order(ast_fields: &[Spanned], checked_fields: &[C out } -fn decorators_metadata(decorators: &[Spanned], checker: &TypeChecker) -> Vec { +fn decorators_metadata( + decorators: &[Spanned], + checker: &TypeChecker, + decorated_callable: Option<&ApiCallableMetadata>, +) -> Vec { decorators .iter() .map(|decorator| { @@ -978,12 +1200,24 @@ fn decorators_metadata(decorators: &[Spanned], checker: &TypeChecker) path: resolved, source_name: decorator.node.path.segments.join("."), anchor: source_span(decorator.span), + type_args: decorator + .node + .type_args + .iter() + .map(|type_arg| { + type_ref_from_resolved(&crate::frontend::symbols::resolve_type( + &type_arg.node, + &checker.symbols, + )) + }) + .collect(), args: decorator .node .args .iter() .map(|arg| decorator_arg_metadata(arg, checker)) .collect(), + decorated_callable: decorated_callable.cloned(), } }) .collect() @@ -1016,12 +1250,134 @@ fn decorator_expr_value(expr: &Spanned, checker: &TypeChecker) -> Decorato name: name.clone(), value: checker.type_info().const_value(name).map(safe_value_from_const), }, + Expr::Field(base, field) => { + let mut path = decorator_expr_path(&base.node); + if path.is_empty() { + DecoratorValue::Unsupported { + reason: "decorator field expression is not a symbolic path".to_string(), + } + } else { + path.push(field.clone()); + DecoratorValue::SymbolRef { path } + } + } + Expr::List(entries) => DecoratorValue::List { + items: entries + .iter() + .map(|entry| match entry { + ListEntry::Element(value) => decorator_expr_value(value, checker), + ListEntry::Spread(value) => DecoratorValue::Unsupported { + reason: format!( + "decorator list spread `{}` is not declaration-safe metadata", + decorator_expr_label(&value.node) + ), + }, + }) + .collect(), + }, + Expr::Dict(entries) => { + let mut metadata_entries = Vec::new(); + for entry in entries { + match entry { + DictEntry::Pair(key, value) => metadata_entries.push(DecoratorDictEntry { + key: decorator_expr_value(key, checker), + value: decorator_expr_value(value, checker), + }), + DictEntry::Spread(value) => metadata_entries.push(DecoratorDictEntry { + key: DecoratorValue::Unsupported { + reason: "decorator dict spread has no declaration-safe key".to_string(), + }, + value: decorator_expr_value(value, checker), + }), + } + } + DecoratorValue::Dict { + entries: metadata_entries, + } + } + Expr::Call(callee, type_args, args) => { + let path = decorator_expr_path(&callee.node); + if path.is_empty() { + return DecoratorValue::Unsupported { + reason: "decorator call callee is not a symbolic path".to_string(), + }; + } + DecoratorValue::Call { + callee: path, + type_args: type_args + .iter() + .map(|type_arg| { + type_ref_from_resolved(&crate::frontend::symbols::resolve_type( + &type_arg.node, + &checker.symbols, + )) + }) + .collect(), + args: args + .iter() + .map(|arg| decorator_call_arg_metadata(arg, checker)) + .collect(), + } + } + Expr::Constructor(name, args) => DecoratorValue::Call { + callee: vec![name.clone()], + type_args: Vec::new(), + args: args + .iter() + .map(|arg| decorator_call_arg_metadata(arg, checker)) + .collect(), + }, _ => DecoratorValue::Unsupported { reason: "decorator argument is not a literal, const reference, or type".to_string(), }, } } +fn decorator_call_arg_metadata(arg: &CallArg, checker: &TypeChecker) -> DecoratorCallArgMetadata { + match arg { + CallArg::Positional(value) => DecoratorCallArgMetadata::Positional { + value: decorator_expr_value(value, checker), + }, + CallArg::Named(name, value) => DecoratorCallArgMetadata::Named { + name: name.clone(), + value: decorator_expr_value(value, checker), + }, + CallArg::PositionalUnpack(value) => DecoratorCallArgMetadata::PositionalUnpack { + value: decorator_expr_value(value, checker), + }, + CallArg::KeywordUnpack(value) => DecoratorCallArgMetadata::KeywordUnpack { + value: decorator_expr_value(value, checker), + }, + } +} + +fn decorator_expr_path(expr: &Expr) -> Vec { + match expr { + Expr::Ident(name) => vec![name.clone()], + Expr::Field(base, field) => { + let mut path = decorator_expr_path(&base.node); + if path.is_empty() { + return Vec::new(); + } + path.push(field.clone()); + path + } + _ => Vec::new(), + } +} + +fn decorator_expr_label(expr: &Expr) -> &'static str { + match expr { + Expr::Ident(_) => "identifier", + Expr::Literal(_) => "literal", + Expr::Call(_, _, _) | Expr::Constructor(_, _) => "call", + Expr::List(_) => "list", + Expr::Dict(_) => "dict", + Expr::Field(_, _) => "field", + _ => "expression", + } +} + /// Convert a literal into the safe metadata subset used by checked API output. fn safe_value_from_literal(literal: &crate::frontend::ast::Literal) -> SafeMetadataValue { match literal { @@ -1861,6 +2217,276 @@ pub def avg(values: List[float]) -> float: Ok(()) } + #[test] + fn checked_api_metadata_preserves_decorated_function_source_signature() -> Result<(), String> { + let source = r#" +def keep(func: (int) -> int) -> (int) -> int: + return func + +@keep +pub def decorated(value: int) -> int: + """Return the input value. + + Args: + value: Input value. + """ + return value +"#; + let metadata = metadata_for(source).map_err(|errs| format!("{errs:?}"))?; + let function = metadata + .declarations + .iter() + .find_map(|decl| match decl { + ApiDeclaration::Function(function) if function.name == "decorated" => Some(function), + _ => None, + }) + .ok_or_else(|| "expected decorated function metadata".to_string())?; + + assert_eq!(function.params.len(), 1); + assert_eq!(function.params[0].name, "value"); + assert_eq!( + function.params[0].ty, + TypeRef::Named { + name: "int".to_string(), + } + ); + + let diagnostics = validate_checked_api_docstrings(&[metadata]); + assert!( + diagnostics.is_empty(), + "expected decorated source signature to satisfy docstring validation, got {diagnostics:?}" + ); + Ok(()) + } + + #[test] + fn checked_api_metadata_preserves_generic_decorator_factory_source_signature() -> Result<(), String> { + let source = r#" +model ColumnExpr: + name: str + +def registered[F](name: str) -> ((F) -> F): + return (func) => func + +@registered("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + """Build a column expression. + + Args: + name: Column name. + """ + return ColumnExpr(name=name) +"#; + let metadata = metadata_for(source).map_err(|errs| format!("{errs:?}"))?; + let function = metadata + .declarations + .iter() + .find_map(|decl| match decl { + ApiDeclaration::Function(function) if function.name == "col" => Some(function), + _ => None, + }) + .ok_or_else(|| "expected decorated function metadata".to_string())?; + + assert_eq!(function.params.len(), 1); + assert_eq!(function.params[0].name, "name"); + assert_eq!( + function.params[0].ty, + TypeRef::Named { + name: "str".to_string(), + } + ); + assert_eq!( + function.return_type, + TypeRef::Named { + name: "ColumnExpr".to_string(), + } + ); + + let diagnostics = validate_checked_api_docstrings(&[metadata]); + assert!( + diagnostics.is_empty(), + "expected generic decorator factory source signature to satisfy docstring validation, got {diagnostics:?}" + ); + Ok(()) + } + + #[test] + fn checked_api_metadata_projects_decorated_callable_context_issue694() -> Result<(), String> { + let source = r#" +const EQUAL_FUNCTION_ANCHOR = "substrait.equal" + +model ColumnExpr: + name: str + +model FunctionLifecycle: + since: str + changed: List[str] + deprecated: Option[str] + +def extension_mapping(name: str, anchor: str) -> str: + return name + +def deterministic_spec(kind: str, lifecycle: FunctionLifecycle, mapping: str) -> str: + return kind + +def registered[F](spec: str) -> ((F) -> F): + return (func) => func + +@registered(deterministic_spec("scalar", FunctionLifecycle(since="v0.3", changed=[], deprecated=None), extension_mapping("equal", EQUAL_FUNCTION_ANCHOR))) +pub def eq(left: ColumnExpr, right: ColumnExpr) -> ColumnExpr: + return left +"#; + let metadata = metadata_for(source).map_err(|errs| format!("{errs:?}"))?; + let function = metadata + .declarations + .iter() + .find_map(|decl| match decl { + ApiDeclaration::Function(function) if function.name == "eq" => Some(function), + _ => None, + }) + .ok_or_else(|| "expected decorated function metadata".to_string())?; + let decorator = function + .decorators + .first() + .ok_or_else(|| "expected decorator metadata".to_string())?; + let callable = decorator + .decorated_callable + .as_ref() + .ok_or_else(|| "expected decorated callable context".to_string())?; + + assert_eq!(callable.name, "eq"); + assert_eq!( + callable + .params + .iter() + .map(|param| (param.name.as_str(), ¶m.ty)) + .collect::>(), + vec![ + ( + "left", + &TypeRef::Named { + name: "ColumnExpr".to_string(), + }, + ), + ( + "right", + &TypeRef::Named { + name: "ColumnExpr".to_string(), + }, + ), + ] + ); + assert_eq!( + callable.return_type, + TypeRef::Named { + name: "ColumnExpr".to_string(), + } + ); + + let [ + DecoratorArgMetadata::Positional { + value: DecoratorValue::Call { callee, args, .. }, + }, + ] = decorator.args.as_slice() + else { + return Err(format!( + "expected structured decorator call metadata, got {decorator:?}" + )); + }; + assert_eq!(callee, &vec!["deterministic_spec".to_string()]); + let lifecycle_args = args + .iter() + .find_map(|arg| match arg { + DecoratorCallArgMetadata::Positional { + value: DecoratorValue::Call { callee, args, .. }, + } if callee == &vec!["FunctionLifecycle".to_string()] => Some(args), + _ => None, + }) + .ok_or_else(|| format!("expected nested lifecycle constructor call metadata, got {args:?}"))?; + assert!( + lifecycle_args.iter().any(|arg| matches!( + arg, + DecoratorCallArgMetadata::Named { + name, + value: DecoratorValue::List { items }, + } if name == "changed" && items.is_empty() + )), + "expected lifecycle `changed=[]` metadata, got {lifecycle_args:?}" + ); + assert!( + lifecycle_args.iter().any(|arg| matches!( + arg, + DecoratorCallArgMetadata::Named { + name, + value: DecoratorValue::Literal { + value: SafeMetadataValue::None, + }, + } if name == "deprecated" + )), + "expected lifecycle `deprecated=None` metadata, got {lifecycle_args:?}" + ); + assert!( + args.iter().any(|arg| matches!( + arg, + DecoratorCallArgMetadata::Positional { + value: DecoratorValue::Call { callee, args, .. }, + } if callee == &vec!["extension_mapping".to_string()] + && args.iter().any(|arg| matches!( + arg, + DecoratorCallArgMetadata::Positional { + value: DecoratorValue::ConstRef { + name, + value: Some(SafeMetadataValue::String(value)), + }, + } if name == "EQUAL_FUNCTION_ANCHOR" && value == "substrait.equal" + )) + )), + "expected nested extension mapping call metadata with checked const ref, got {args:?}" + ); + Ok(()) + } + + #[test] + fn checked_api_metadata_rejects_non_symbolic_decorator_field_metadata() -> Result<(), String> { + let source = r#" +model Holder: + value: str + +def holder() -> Holder: + return Holder(value="equal") + +def registered[F](name: str) -> ((F) -> F): + return (func) => func + +@registered(holder().value) +pub def eq(left: int, right: int) -> int: + return left +"#; + let metadata = metadata_for(source).map_err(|errs| format!("{errs:?}"))?; + let function = metadata + .declarations + .iter() + .find_map(|decl| match decl { + ApiDeclaration::Function(function) if function.name == "eq" => Some(function), + _ => None, + }) + .ok_or_else(|| "expected decorated function metadata".to_string())?; + let [ + DecoratorArgMetadata::Positional { + value: DecoratorValue::Unsupported { reason }, + }, + ] = function.decorators[0].args.as_slice() + else { + return Err(format!( + "expected non-symbolic field decorator argument to stay unsupported, got {:?}", + function.decorators[0].args + )); + }; + + assert_eq!(reason, "decorator field expression is not a symbolic path"); + Ok(()) + } + #[test] fn checked_api_docstring_validation_matches_overloaded_method_by_params() -> Result<(), String> { let source = r#" diff --git a/src/frontend/ast_walk.rs b/src/frontend/ast_walk.rs index 113ad3d4e..a591facea 100644 --- a/src/frontend/ast_walk.rs +++ b/src/frontend/ast_walk.rs @@ -376,7 +376,7 @@ where }), Expr::FString(parts) => parts.iter().any(|part| match part { crate::frontend::ast::FStringPart::Literal(_) => false, - crate::frontend::ast::FStringPart::Expr(expr) => expr_has(&expr.node, pred), + crate::frontend::ast::FStringPart::Expr { expr, .. } => expr_has(&expr.node, pred), }), Expr::Yield(Some(expr)) => expr_has(&expr.node, pred), Expr::Yield(None) | Expr::Partial(_) => false, diff --git a/src/frontend/library_exports.rs b/src/frontend/library_exports.rs index 16bf18693..b7cafeb64 100644 --- a/src/frontend/library_exports.rs +++ b/src/frontend/library_exports.rs @@ -6,9 +6,12 @@ use std::collections::HashMap; use crate::frontend::ast::{ - AliasDecl, ClassDecl, Declaration, DictEntry, EnumDecl, Expr, FunctionDecl, ListEntry, Literal, ModelDecl, - NewtypeDecl, PartialDecl, Program, Spanned, TraitBound, TraitDecl, TypeAliasDecl, TypeParam, Visibility, + AliasDecl, ClassDecl, Declaration, DictEntry, EnumDecl, Expr, FunctionDecl, ImportDecl, ImportItem, ImportKind, + ListEntry, Literal, ModelDecl, NewtypeDecl, PartialDecl, Program, Spanned, TraitBound, TraitDecl, TypeAliasDecl, + TypeParam, Visibility, }; +use crate::frontend::decorator_resolution; +use crate::frontend::module::canonicalize_source_module_segments; use crate::frontend::symbols::{ CallableParam, ClassInfo, FieldInfo, FunctionInfo, MethodInfo, ModelInfo, NewtypeInfo, ResolvedType, SymbolKind, TraitInfo, TypeBoundInfo, TypeInfo, ValueEnumBacking, ValueEnumValue, VariableInfo, resolve_type, @@ -117,6 +120,7 @@ pub struct CheckedTypeAliasExport { pub struct CheckedAliasExport { pub name: String, pub target_path: Vec, + pub projected_function: Option, } #[derive(Debug, Clone)] @@ -306,6 +310,9 @@ pub fn collect_checked_public_exports(program: &Program, checker: &TypeChecker) exports.push(export); } } + Declaration::Import(import) if matches!(import.visibility, Visibility::Public) => { + exports.extend(checked_import_exports(import, checker)); + } Declaration::Partial(partial) if matches!(partial.visibility, Visibility::Public) => { if let Some(export) = checked_partial_export(partial, checker) { exports.push(CheckedNamedExport { @@ -324,16 +331,97 @@ pub fn collect_checked_public_exports(program: &Program, checker: &TypeChecker) /// Build a checked public export entry for a module-level alias. fn checked_alias_export(alias: &AliasDecl, checker: &TypeChecker) -> Option { - checker.lookup_symbol(alias.name.as_str())?; + let symbol = checker.lookup_symbol(alias.name.as_str())?; + let projected_function = checked_projected_function_export(&alias.name, &symbol.kind); Some(CheckedNamedExport { name: alias.name.clone(), kind: CheckedExportKind::Alias(CheckedAliasExport { name: alias.name.clone(), target_path: alias.target.segments.clone(), + projected_function, }), }) } +fn checked_import_exports(import: &ImportDecl, checker: &TypeChecker) -> Vec { + match &import.kind { + ImportKind::From { module, items } => { + let base_path = + canonicalize_source_module_segments(&decorator_resolution::path_segments_with_prefix(module)); + checked_import_item_exports(items, base_path, checker) + } + ImportKind::RustFrom { + crate_name, + path, + items, + .. + } => { + let mut base_path = vec!["rust".to_string(), crate_name.clone()]; + base_path.extend(path.iter().cloned()); + checked_import_item_exports(items, base_path, checker) + } + ImportKind::PubFrom { library, items } => { + let base_path = vec!["pub".to_string(), library.clone()]; + checked_import_item_exports(items, base_path, checker) + } + _ => Vec::new(), + } +} + +fn checked_import_item_exports( + items: &[ImportItem], + base_path: Vec, + checker: &TypeChecker, +) -> Vec { + items + .iter() + .map(|item| { + let exported_name = item.alias.as_ref().unwrap_or(&item.name).clone(); + let mut target_path = base_path.clone(); + target_path.push(item.name.clone()); + let projected_function = checker + .lookup_symbol(exported_name.as_str()) + .and_then(|symbol| checked_projected_function_export(&exported_name, &symbol.kind)); + CheckedNamedExport { + name: exported_name.clone(), + kind: CheckedExportKind::Alias(CheckedAliasExport { + name: exported_name, + target_path, + projected_function, + }), + } + }) + .collect() +} + +/// Build manifest-ready callable metadata for an alias that projects a function. +fn checked_alias_function_export(name: &str, info: &FunctionInfo) -> CheckedFunctionExport { + CheckedFunctionExport { + name: name.to_string(), + type_params: checked_function_type_params(info), + params: info.params.clone(), + return_type: info.return_type.clone(), + is_async: info.is_async, + } +} + +fn checked_projected_function_export(name: &str, kind: &SymbolKind) -> Option { + match kind { + SymbolKind::Function(info) => Some(checked_alias_function_export(name, info)), + SymbolKind::Variable(VariableInfo { + ty: ResolvedType::Function(params, return_type), + .. + }) => Some(CheckedFunctionExport { + name: name.to_string(), + type_params: Vec::new(), + params: params.clone(), + return_type: return_type.as_ref().clone(), + is_async: false, + }), + _ => None, + } +} + /// Build checked export metadata for a public partial callable preset. fn checked_partial_export(partial: &PartialDecl, checker: &TypeChecker) -> Option { let symbol = checker.lookup_symbol(partial.name.as_str())?; @@ -482,6 +570,9 @@ fn checked_preset_path(expr: &Expr) -> Vec { Expr::Ident(name) => vec![name.clone()], Expr::Field(base, field) => { let mut path = checked_preset_path(&base.node); + if path.is_empty() { + return Vec::new(); + } path.push(field.clone()); path } @@ -519,6 +610,20 @@ fn checked_function_export(function: &FunctionDecl, checker: &TypeChecker) -> Op }) } +/// Convert checked function metadata type parameters into export metadata type parameters. +fn checked_function_type_params(info: &FunctionInfo) -> Vec { + info.type_params + .iter() + .map(|name| CheckedTypeParam { + name: name.clone(), + bounds: info + .type_param_bound_details + .get(name) + .map_or_else(Vec::new, |bounds| map_type_bound_infos(bounds)), + }) + .collect() +} + fn checked_type_alias_export(alias: &TypeAliasDecl, checker: &TypeChecker) -> CheckedTypeAliasExport { let target = resolve_type(&alias.target.node, &checker.symbols); CheckedTypeAliasExport { @@ -866,3 +971,27 @@ fn sorted_vec(mut values: Vec) -> Vec { values.sort(); values } + +#[cfg(test)] +mod tests { + use super::*; + use crate::frontend::ast::{Span, Spanned}; + + fn spanned(expr: Expr) -> Spanned { + Spanned::new(expr, Span::default()) + } + + #[test] + fn checked_preset_value_rejects_non_symbolic_field_paths() { + let value = Expr::Field( + Box::new(spanned(Expr::Call( + Box::new(spanned(Expr::Ident("defaults".to_string()))), + Vec::new(), + Vec::new(), + ))), + "method".to_string(), + ); + + assert_eq!(checked_preset_value(&value), CheckedPresetValue::Unsupported); + } +} diff --git a/src/frontend/module.rs b/src/frontend/module.rs index 1130b9104..978c05fd6 100644 --- a/src/frontend/module.rs +++ b/src/frontend/module.rs @@ -455,6 +455,11 @@ pub fn exported_symbols(ast: &Program) -> Vec { exports.push(ExportedSymbol::Function(f.name.clone())); } } + Declaration::Partial(p) => { + if matches!(p.visibility, Visibility::Public) { + exports.push(ExportedSymbol::Function(p.name.clone())); + } + } Declaration::Import(import) => { // Both `from module import X` and `from rust::crate import X` are treated as re-exports. This lets // stdlib files like `response.incn` expose axum types (`from rust::axum import Json`) to importers @@ -472,7 +477,7 @@ pub fn exported_symbols(ast: &Program) -> Vec { } } } - Declaration::Partial(_) | Declaration::Docstring(_) | Declaration::TestModule(_) => {} + Declaration::Docstring(_) | Declaration::TestModule(_) => {} } } @@ -1091,6 +1096,26 @@ source-root = "library" } } + #[test] + fn test_exported_symbols_partial() -> Result<(), Vec> { + let source = r#" +pub def route(method: str, path: str) -> str: + return path + +pub get = partial route(method="GET") +"#; + let tokens = lexer::lex(source).map_err(|e| e.iter().map(|x| x.message.clone()).collect::>())?; + let ast = parser::parse(&tokens).map_err(|e| e.iter().map(|x| x.message.clone()).collect::>())?; + let exports = exported_symbols(&ast); + assert!( + exports + .iter() + .any(|export| matches!(export, ExportedSymbol::Function(name) if name == "get")), + "expected public partial callable export, got {exports:?}" + ); + Ok(()) + } + #[test] fn test_exported_symbols_ignores_module_imports() { let import = ImportDecl { diff --git a/src/frontend/testing_markers.rs b/src/frontend/testing_markers.rs index c96ef6647..626345ba4 100644 --- a/src/frontend/testing_markers.rs +++ b/src/frontend/testing_markers.rs @@ -135,24 +135,10 @@ pub struct TestingFixtureMarkerArgs { } impl Default for TestingMarkerSemantics { - /// Return the built-in marker semantics used as the extraction baseline for stdlib metadata. + /// Return fixture defaults used while strict marker metadata is loaded from stdlib source. fn default() -> Self { - let mut marker_kinds = HashMap::new(); - marker_kinds.insert("test".to_string(), TestingMarkerKind::Test); - marker_kinds.insert("fixture".to_string(), TestingMarkerKind::Fixture); - marker_kinds.insert("skip".to_string(), TestingMarkerKind::Skip); - marker_kinds.insert("skipif".to_string(), TestingMarkerKind::SkipIf); - marker_kinds.insert("xfail".to_string(), TestingMarkerKind::XFail); - marker_kinds.insert("xfailif".to_string(), TestingMarkerKind::XFailIf); - marker_kinds.insert("slow".to_string(), TestingMarkerKind::Slow); - marker_kinds.insert("mark".to_string(), TestingMarkerKind::Mark); - marker_kinds.insert("resource".to_string(), TestingMarkerKind::Resource); - marker_kinds.insert("serial".to_string(), TestingMarkerKind::Serial); - marker_kinds.insert("timeout".to_string(), TestingMarkerKind::Timeout); - marker_kinds.insert("parametrize".to_string(), TestingMarkerKind::Parametrize); - Self { - marker_kinds, + marker_kinds: HashMap::new(), fixture_scope_arg: "scope".to_string(), fixture_autouse_arg: "autouse".to_string(), fixture_scope_function: "function".to_string(), @@ -358,9 +344,48 @@ fn extract_testing_marker_semantics(program: &ast::Program) -> Result Result<(), TestingMarkerLoadError> { + let expected_names = incan_core::lang::testing::RUNNER_ONLY_MARKER_NAMES; + let mut missing = Vec::new(); + let mut mismatched = Vec::new(); + + for expected_name in expected_names { + let Some(actual_kind) = semantics.marker_kinds.get(*expected_name) else { + missing.push(*expected_name); + continue; + }; + let expected_kind = TestingMarkerKind::from_str(expected_name).ok_or_else(|| { + TestingMarkerLoadError::new(format!( + "runtime marker inventory contains unknown marker `{expected_name}`" + )) + })?; + if actual_kind != &expected_kind { + mismatched.push(format!( + "{expected_name} declares {actual_kind:?}, expected {expected_kind:?}" + )); + } + } + + let unexpected = semantics + .marker_kinds + .keys() + .filter(|name| !expected_names.contains(&name.as_str())) + .cloned() + .collect::>(); + + if !missing.is_empty() || !unexpected.is_empty() || !mismatched.is_empty() { + return Err(TestingMarkerLoadError::new(format!( + "std.testing marker metadata does not match runtime marker inventory; missing={missing:?}, unexpected={unexpected:?}, mismatched={mismatched:?}" + ))); + } + + Ok(()) +} + #[derive(Debug, Clone, PartialEq, Eq)] struct TestingMarkerAnnotation { kind: TestingMarkerKind, @@ -399,6 +424,7 @@ fn parse_testing_metadata_dict( }; let mut kind: Option = None; + let mut runner_only = false; let mut fixture_scope_arg: Option = None; let mut fixture_autouse_arg: Option = None; let mut fixture_scopes: Option<[String; 3]> = None; @@ -428,10 +454,13 @@ fn parse_testing_metadata_dict( }; kind = Some(parsed_kind); } - TESTING_MARKER_RUNNER_ONLY_KEY if expr_as_bool_literal(value_expr).is_none() => { - return Err(TestingMarkerLoadError::new( - "malformed runner_only metadata value (expected bool)", - )); + TESTING_MARKER_RUNNER_ONLY_KEY => { + let Some(value) = expr_as_bool_literal(value_expr) else { + return Err(TestingMarkerLoadError::new( + "malformed runner_only metadata value (expected bool)", + )); + }; + runner_only = value; } TESTING_FIXTURE_SCOPE_ARG_KEY => { let Some(value) = expr_as_string_literal(value_expr) else { @@ -466,6 +495,12 @@ fn parse_testing_metadata_dict( return Ok(None); }; + if !runner_only { + return Err(TestingMarkerLoadError::new( + "std.testing marker metadata must declare runner_only=true", + )); + } + Ok(Some(TestingMarkerAnnotation { kind, fixture_scope_arg, @@ -537,6 +572,19 @@ mod tests { Ok(()) } + #[test] + fn test_std_testing_metadata_matches_runtime_marker_names() -> Result<(), Box> { + let semantics = load_testing_marker_semantics_from_stdlib()?; + let mut metadata_names: Vec<&str> = semantics.marker_kinds.keys().map(String::as_str).collect(); + metadata_names.sort_unstable(); + + let mut runtime_names = incan_core::lang::testing::RUNNER_ONLY_MARKER_NAMES.to_vec(); + runtime_names.sort_unstable(); + + assert_eq!(metadata_names, runtime_names); + Ok(()) + } + #[test] fn test_testing_marker_semantics_malformed_annotation_is_error() -> Result<(), Box> { let source = r#" @@ -561,4 +609,82 @@ def xfail(reason: str = "") -> None: assert!(extracted.is_err(), "malformed marker annotation should fail extraction"); Ok(()) } + + #[test] + fn test_testing_marker_semantics_rejects_non_runner_only_marker() -> Result<(), Box> { + let source = r#" +@rust.extern(metadata={"marker_kind": "skip", "runner_only": false}) +def skip(reason: str = "") -> None: + ... +"#; + let tokens = match crate::frontend::lexer::lex(source) { + Ok(tokens) => tokens, + Err(errs) => return Err(format!("lex failed for non-runner-only annotation fixture: {errs:?}").into()), + }; + let program = match crate::frontend::parser::parse(&tokens) { + Ok(program) => program, + Err(errs) => return Err(format!("parse failed for non-runner-only annotation fixture: {errs:?}").into()), + }; + + let extracted = extract_testing_marker_semantics(&program); + assert!( + extracted + .as_ref() + .is_err_and(|err| err.to_string().contains("runner_only=true")), + "non-runner-only marker annotation should fail extraction; got: {extracted:?}" + ); + Ok(()) + } + + #[test] + fn test_testing_marker_semantics_rejects_incomplete_marker_inventory() -> Result<(), Box> { + let source = r#" +@rust.extern(metadata={"marker_kind": "skip", "runner_only": true}) +def skip(reason: str = "") -> None: + ... +"#; + let tokens = match crate::frontend::lexer::lex(source) { + Ok(tokens) => tokens, + Err(errs) => return Err(format!("lex failed for incomplete marker inventory fixture: {errs:?}").into()), + }; + let program = match crate::frontend::parser::parse(&tokens) { + Ok(program) => program, + Err(errs) => return Err(format!("parse failed for incomplete marker inventory fixture: {errs:?}").into()), + }; + + let extracted = extract_testing_marker_semantics(&program); + assert!( + extracted + .as_ref() + .is_err_and(|err| err.to_string().contains("runtime marker inventory")), + "incomplete marker inventory should fail extraction; got: {extracted:?}" + ); + Ok(()) + } + + #[test] + fn test_testing_marker_semantics_rejects_function_kind_mismatch() -> Result<(), Box> { + let source = r#" +@rust.extern(metadata={"marker_kind": "xfail", "runner_only": true}) +def skip(reason: str = "") -> None: + ... +"#; + let tokens = match crate::frontend::lexer::lex(source) { + Ok(tokens) => tokens, + Err(errs) => return Err(format!("lex failed for mismatched marker fixture: {errs:?}").into()), + }; + let program = match crate::frontend::parser::parse(&tokens) { + Ok(program) => program, + Err(errs) => return Err(format!("parse failed for mismatched marker fixture: {errs:?}").into()), + }; + + let extracted = extract_testing_marker_semantics(&program); + assert!( + extracted + .as_ref() + .is_err_and(|err| err.to_string().contains("mismatched")), + "mismatched marker inventory should fail extraction; got: {extracted:?}" + ); + Ok(()) + } } diff --git a/src/frontend/typechecker/check_decl.rs b/src/frontend/typechecker/check_decl.rs index 8e2c48a27..cfb384965 100644 --- a/src/frontend/typechecker/check_decl.rs +++ b/src/frontend/typechecker/check_decl.rs @@ -11,7 +11,7 @@ use crate::frontend::testing_markers::{ TestingFixtureMarkerArgs, TestingMarkerSemantics, load_testing_marker_semantics, resolve_testing_fixture_marker_args, }; -use crate::frontend::typechecker::helpers::{dict_ty, list_ty}; +use crate::frontend::typechecker::helpers::{collection_type_id, dict_ty, list_ty}; use super::{DecoratedFunctionBindingInfo, DecoratedMethodBindingInfo, TestingFixtureInfo, TypeChecker, YieldContext}; use incan_core::interop::{RustItemKind, RustItemMetadata, RustTraitAssoc}; @@ -19,7 +19,9 @@ use incan_core::lang::decorators::{self, DecoratorId}; use incan_core::lang::derives::{self, DeriveId}; use incan_core::lang::magic_methods; use incan_core::lang::stdlib; +use incan_core::lang::testing; use incan_core::lang::traits::{self as builtin_traits, TraitId}; +use incan_core::lang::types::collections::CollectionTypeId; use incan_semantics_core::SurfaceModifierTypeCheck; use std::collections::{HashMap, HashSet}; @@ -172,7 +174,10 @@ fn fixture_function_span(func: &FunctionDecl) -> Span { /// Return whether a decorator resolves to the RFC 004 `std.testing.fixture` marker path. fn is_possible_testing_fixture_decorator(dec: &Decorator, aliases: &HashMap>) -> bool { let resolved = crate::frontend::decorator_resolution::resolve_decorator_path(dec, aliases); - resolved.len() == 3 && resolved[0] == "std" && resolved[1] == "testing" && resolved[2] == "fixture" + resolved.len() == 3 + && resolved[0] == stdlib::STDLIB_ROOT + && resolved[1] == testing::STDLIB_TESTING_MODULE + && resolved[2] == testing::TESTING_MARKER_FIXTURE } /// Return whether any declaration in this slice of AST may be a `std.testing.fixture`. @@ -2439,6 +2444,7 @@ impl TypeChecker { // Define fields in scope for field in &model.fields { let ty = self.resolve_type_checked(&field.node.ty); + self.validate_direct_recursive_model_field(&model.name, &ty, field.span); self.symbols.define(Symbol { name: field.node.name.clone(), kind: SymbolKind::Field(FieldInfo { @@ -2485,6 +2491,129 @@ impl TypeChecker { self.symbols.exit_scope(); } + /// Reject model fields whose resolved type contains the model itself without an indirection boundary. + fn validate_direct_recursive_model_field(&mut self, model_name: &str, field_ty: &ResolvedType, span: Span) { + let mut visiting = HashSet::new(); + if self.type_contains_direct_recursive_model(field_ty, model_name, &mut visiting) { + self.errors.push(CompileError::type_error( + format!( + "Model '{model_name}' has a direct recursive field type '{field_ty}'. Use an indirection such as List[...] for recursive payloads." + ), + span, + )); + } + } + + /// Return whether a type contains the target model through only inline Rust-layout positions. + fn type_contains_direct_recursive_model( + &self, + ty: &ResolvedType, + model_name: &str, + visiting: &mut HashSet, + ) -> bool { + match ty { + ResolvedType::Named(name) => { + self.nominal_type_contains_direct_recursive_model(name, &[], model_name, visiting) + } + ResolvedType::Generic(name, args) if name == UNION_TYPE_NAME => args + .iter() + .any(|arg| self.type_contains_direct_recursive_model(arg, model_name, visiting)), + ResolvedType::Generic(name, args) => match collection_type_id(name.as_str()) { + Some( + CollectionTypeId::List + | CollectionTypeId::Dict + | CollectionTypeId::Set + | CollectionTypeId::FrozenList + | CollectionTypeId::FrozenDict + | CollectionTypeId::FrozenSet + | CollectionTypeId::Generator, + ) => false, + Some(CollectionTypeId::Tuple | CollectionTypeId::Option | CollectionTypeId::Result) => args + .iter() + .any(|arg| self.type_contains_direct_recursive_model(arg, model_name, visiting)), + None => self.nominal_type_contains_direct_recursive_model(name, args, model_name, visiting), + }, + ResolvedType::Tuple(items) => items + .iter() + .any(|item| self.type_contains_direct_recursive_model(item, model_name, visiting)), + ResolvedType::Ref(_) + | ResolvedType::RefMut(_) + | ResolvedType::FrozenList(_) + | ResolvedType::FrozenDict(_, _) + | ResolvedType::FrozenSet(_) + | ResolvedType::Function(_, _) => false, + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Numeric(_) + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + | ResolvedType::TypeVar(_) + | ResolvedType::SelfType + | ResolvedType::RustPath(_) + | ResolvedType::CallSiteInfer + | ResolvedType::Unknown => false, + } + } + + /// Follow known nominal field types to find direct recursive model layouts. + fn nominal_type_contains_direct_recursive_model( + &self, + type_name: &str, + type_args: &[ResolvedType], + model_name: &str, + visiting: &mut HashSet, + ) -> bool { + if type_name == model_name { + return true; + } + + let visit_key = if type_args.is_empty() { + type_name.to_string() + } else { + format!( + "{}[{}]", + type_name, + type_args.iter().map(ToString::to_string).collect::>().join(", ") + ) + }; + if !visiting.insert(visit_key.clone()) { + return false; + } + + let result = match self.lookup_semantic_type_info(type_name) { + Some(TypeInfo::Model(info)) => { + let subst = type_param_subst_map(&info.type_params, type_args); + info.fields.values().any(|field| { + let field_ty = substitute_resolved_type(&field.ty, &subst); + let field_ty = self.expand_type_aliases(field_ty); + self.type_contains_direct_recursive_model(&field_ty, model_name, visiting) + }) + } + Some(TypeInfo::Class(info)) => { + let subst = type_param_subst_map(&info.type_params, type_args); + info.fields.values().any(|field| { + let field_ty = substitute_resolved_type(&field.ty, &subst); + let field_ty = self.expand_type_aliases(field_ty); + self.type_contains_direct_recursive_model(&field_ty, model_name, visiting) + }) + } + Some(TypeInfo::Newtype(info)) => { + let subst = type_param_subst_map(&info.type_params, type_args); + let underlying = substitute_resolved_type(&info.underlying, &subst); + let underlying = self.expand_type_aliases(underlying); + self.type_contains_direct_recursive_model(&underlying, model_name, visiting) + } + Some(TypeInfo::Enum(_) | TypeInfo::Builtin | TypeInfo::TypeAlias) | None => false, + }; + + visiting.remove(&visit_key); + result + } + fn check_validate_derive_model(&mut self, model: &ModelDecl) { // Validate that validate() exists and has the expected signature. let Some(TypeInfo::Model(info)) = self.lookup_type_info(&model.name) else { @@ -3649,7 +3778,7 @@ impl TypeChecker { return; }; - let mut binding_ty = original_ty; + let mut binding_ty = original_ty.clone(); for decorator in func.decorators.iter().rev() { if self.is_user_defined_decorator_candidate(&decorator.node) { binding_ty = self.apply_user_defined_decorator(decorator, binding_ty, &func.name); @@ -3661,7 +3790,10 @@ impl TypeChecker { { self.type_info.declarations.decorated_function_bindings.insert( func.name.clone(), - DecoratedFunctionBindingInfo { ty: binding_ty.clone() }, + DecoratedFunctionBindingInfo { + ty: binding_ty.clone(), + original_ty, + }, ); symbol.kind = SymbolKind::Variable(VariableInfo { ty: binding_ty, @@ -3806,17 +3938,20 @@ impl TypeChecker { let base = Self::decorator_path_expr_from_import_path(&base_path, decorator.span); let method = path.last().cloned().unwrap_or_default(); Spanned::new( - Expr::MethodCall(Box::new(base), method, Vec::new(), args), + Expr::MethodCall(Box::new(base), method, decorator.node.type_args.clone(), args), decorator.span, ) } else { let callee = Self::decorator_path_expr(&decorator.node, decorator.span); - Spanned::new(Expr::Call(Box::new(callee), Vec::new(), args), decorator.span) + Spanned::new( + Expr::Call(Box::new(callee), decorator.node.type_args.clone(), args), + decorator.span, + ) }; self.check_expr(&factory_expr) } - /// Apply a callable decorator value to the decorated binding type and return the post-decoration binding type. + /// Apply a callable decorator value to the decorated binding type and return the post-decoration callable type. fn apply_decorator_callable( &mut self, display: &str, @@ -3845,7 +3980,12 @@ impl TypeChecker { if self.errors.len() != error_count { return ResolvedType::Unknown; } - substitute_resolved_type(&ret, &type_bindings) + let result_ty = substitute_resolved_type(&ret, &type_bindings); + if !matches!(result_ty, ResolvedType::Function(_, _) | ResolvedType::Unknown) { + self.errors.push(errors::decorator_result_not_callable(display, span)); + return ResolvedType::Unknown; + } + result_ty } /// Convert decorator arguments into ordinary call arguments for user-defined decorator factory checking. @@ -3872,7 +4012,11 @@ impl TypeChecker { fn decorator_display(decorator: &Decorator) -> String { let path = decorator.path.segments.join("."); if decorator.is_call { - format!("{path}(...)") + if decorator.type_args.is_empty() { + format!("{path}(...)") + } else { + format!("{path}[...](...)") + } } else { path } diff --git a/src/frontend/typechecker/check_expr/access.rs b/src/frontend/typechecker/check_expr/access.rs index b41eb0351..35f8ec9b0 100644 --- a/src/frontend/typechecker/check_expr/access.rs +++ b/src/frontend/typechecker/check_expr/access.rs @@ -13,14 +13,14 @@ use crate::frontend::typechecker::helpers::{ option_ty, string_method_return, }; use crate::frontend::typechecker::type_info::{RustMethodTraitImportUse, RustTraitImportInfo}; -use incan_core::interop::{RustCollectionFamily, RustItemKind}; +use incan_core::interop::{RustCollectionFamily, RustFunctionSig, RustItemKind, metadata_free_method_signature}; use incan_core::lang::magic_methods; use incan_core::lang::surface::collection_helpers::{self, BuiltinCollectionHelperId}; use incan_core::lang::surface::types as surface_types; use incan_core::lang::surface::types::{SEMAPHORE_ACQUIRE_ERROR_TYPE_NAME, SEMAPHORE_PERMIT_TYPE_NAME, SurfaceTypeId}; use incan_core::lang::surface::{ dict_methods, float_methods, frozen_bytes_methods, frozen_dict_methods, frozen_list_methods, frozen_set_methods, - list_methods, result_methods, set_methods, + iterator_methods, list_methods, result_methods, set_methods, }; use incan_core::lang::traits::{self as core_traits, TraitId}; use incan_core::lang::types::collections::CollectionTypeId; @@ -63,7 +63,17 @@ fn rust_receiver_display(path: &str) -> String { impl TypeChecker { /// Return whether `method` names an RFC 070 `Result[T, E]` combinator. fn result_combinator_name(method: &str) -> bool { - result_methods::from_str(method).is_some() + matches!( + result_methods::from_str(method), + Some( + result_methods::ResultMethodId::Map + | result_methods::ResultMethodId::MapErr + | result_methods::ResultMethodId::AndThen + | result_methods::ResultMethodId::OrElse + | result_methods::ResultMethodId::Inspect + | result_methods::ResultMethodId::InspectErr + ) + ) } /// Resolve a callable function or callable object to its parameter and return types. @@ -200,6 +210,7 @@ impl TypeChecker { self.validate_result_combinator_callback(method, callback_ty, &err_ty, Some(&ResolvedType::Unit), span); ResolvedType::Generic("Result".to_string(), vec![ok_ty, err_ty]) } + result_methods::ResultMethodId::Unwrap | result_methods::ResultMethodId::UnwrapOr => ResolvedType::Unknown, } } @@ -574,13 +585,15 @@ impl TypeChecker { let iterator_elem = self .iterator_protocol_element_type(base_ty) .unwrap_or_else(|| elem.clone()); + let method_id = iterator_methods::from_str(method)?; + use iterator_methods::IteratorMethodId as M; - match method { - "iter" => { + match method_id { + M::Iter => { self.validate_iterator_method_arity(method, 0, args.len(), span); Some(Self::iterator_protocol_ty(elem)) } - "map" => { + M::Map => { if !self.validate_iterator_method_arity(method, 1, args.len(), span) { return Some(Self::iterator_protocol_ty(ResolvedType::Unknown)); } @@ -591,7 +604,7 @@ impl TypeChecker { ); Some(Self::iterator_protocol_ty(mapped)) } - "filter" | "take_while" | "skip_while" => { + M::Filter | M::TakeWhile | M::SkipWhile => { if self.validate_iterator_method_arity(method, 1, args.len(), span) { self.validate_iterator_callback_return( method, @@ -603,7 +616,7 @@ impl TypeChecker { } Some(Self::iterator_protocol_ty(iterator_elem)) } - "flat_map" => { + M::FlatMap => { if !self.validate_iterator_method_arity(method, 1, args.len(), span) { return Some(Self::iterator_protocol_ty(ResolvedType::Unknown)); } @@ -628,7 +641,7 @@ impl TypeChecker { }; Some(Self::iterator_protocol_ty(flat_elem)) } - "take" | "skip" => { + M::Take | M::Skip => { if self.validate_iterator_method_arity(method, 1, args.len(), span) && let Some(arg_ty) = arg_types.first() && !self.types_compatible(arg_ty, &ResolvedType::Int) @@ -638,7 +651,7 @@ impl TypeChecker { } Some(Self::iterator_protocol_ty(iterator_elem)) } - "chain" => { + M::Chain => { if self.validate_iterator_method_arity(method, 1, args.len(), span) && let Some(arg_ty) = arg_types.first() { @@ -650,14 +663,14 @@ impl TypeChecker { } Some(Self::iterator_protocol_ty(iterator_elem)) } - "enumerate" => { + M::Enumerate => { self.validate_iterator_method_arity(method, 0, args.len(), span); Some(Self::iterator_protocol_ty(ResolvedType::Tuple(vec![ ResolvedType::Int, iterator_elem, ]))) } - "zip" => { + M::Zip => { if !self.validate_iterator_method_arity(method, 1, args.len(), span) { return Some(Self::iterator_protocol_ty(ResolvedType::Unknown)); } @@ -682,7 +695,7 @@ impl TypeChecker { other_elem, ]))) } - "batch" => { + M::Batch => { if self.validate_iterator_method_arity(method, 1, args.len(), span) && let Some(arg_ty) = arg_types.first() && !self.types_compatible(arg_ty, &ResolvedType::Int) @@ -693,15 +706,15 @@ impl TypeChecker { self.validate_iterator_batch_size_literal(args, span); Some(Self::iterator_protocol_ty(list_ty(iterator_elem))) } - "collect" => { + M::Collect => { self.validate_iterator_method_arity(method, 0, args.len(), span); Some(list_ty(iterator_elem)) } - "count" => { + M::Count => { self.validate_iterator_method_arity(method, 0, args.len(), span); Some(ResolvedType::Int) } - "any" | "all" => { + M::Any | M::All => { if self.validate_iterator_method_arity(method, 1, args.len(), span) { self.validate_iterator_callback_return( method, @@ -713,7 +726,7 @@ impl TypeChecker { } Some(ResolvedType::Bool) } - "find" => { + M::Find => { if self.validate_iterator_method_arity(method, 1, args.len(), span) { self.validate_iterator_callback_return( method, @@ -725,7 +738,7 @@ impl TypeChecker { } Some(option_ty(iterator_elem)) } - "reduce" | "fold" => { + M::Reduce | M::Fold => { if !self.validate_iterator_method_arity(method, 2, args.len(), span) { return Some(ResolvedType::Unknown); } @@ -739,7 +752,7 @@ impl TypeChecker { ); Some(acc_ty) } - "for_each" => { + M::ForEach => { if self.validate_iterator_method_arity(method, 1, args.len(), span) { self.validate_iterator_callback_return( method, @@ -751,11 +764,10 @@ impl TypeChecker { } Some(ResolvedType::Unit) } - "sum" => { + M::Sum => { self.validate_iterator_method_arity(method, 0, args.len(), span); Some(self.iterator_sum_output_type(&iterator_elem, span)) } - _ => None, } } @@ -1342,7 +1354,33 @@ impl TypeChecker { if preserves_lookup_arg_shape { self.type_info.record_regular_method_arg_shape(receiver_span, method); } - let metadata = self.rust_item_metadata_for_path(rust_path)?; + let Some(metadata) = self.rust_item_metadata_for_path(rust_path) else { + if let Some(import_use) = self.record_unique_rust_trait_import_for_unresolved_receiver_call(method, span) + && let Some(sig) = import_use.signature.as_ref() + { + let callable_display = format!("rust::{rust_path}.{method}"); + let ret = self.validate_rust_method_call( + callable_display.as_str(), + sig, + args, + arg_types, + preserves_lookup_arg_shape, + span, + ); + return Some(Self::substitute_rust_self_type(ret, rust_path)); + } + if let Some(ret) = self.validate_metadata_free_rust_method_call( + rust_path, + method, + args, + arg_types, + preserves_lookup_arg_shape, + span, + ) { + return Some(ret); + } + return None; + }; match &metadata.kind { RustItemKind::Type(_) => { let Some(sig) = self.rust_method_signature(rust_path, method) else { @@ -1360,6 +1398,16 @@ impl TypeChecker { ); return Some(Self::substitute_rust_self_type(ret, rust_path)); } + if let Some(ret) = self.validate_metadata_free_rust_method_call( + rust_path, + method, + args, + arg_types, + preserves_lookup_arg_shape, + span, + ) { + return Some(ret); + } // Stay permissive when no unambiguous imported trait or trait method signature can be selected. return Some(ResolvedType::Unknown); }; @@ -1396,6 +1444,34 @@ impl TypeChecker { } } + /// Validate one metadata-free Rust method compatibility rule through the ordinary Rust-boundary path. + fn validate_metadata_free_rust_method_call( + &mut self, + rust_path: &str, + method: &str, + args: &[CallArg], + arg_types: &[ResolvedType], + preserves_lookup_arg_shape: bool, + span: Span, + ) -> Option { + let sig: RustFunctionSig = metadata_free_method_signature(rust_path, method)?; + let callable_display = format!("rust::{rust_path}.{method}"); + let error_count = self.errors.len(); + let ret = self.validate_rust_method_call( + callable_display.as_str(), + &sig, + args, + arg_types, + preserves_lookup_arg_shape, + span, + ); + if self.errors.len() > error_count { + Some(ResolvedType::Unknown) + } else { + Some(Self::substitute_rust_self_type(ret, rust_path)) + } + } + /// Record the imported Rust extension trait needed for a method call when metadata proves a unique match. /// /// Rust method lookup needs the trait binding in scope even though the emitted call remains `receiver.method(...)`. @@ -1434,6 +1510,37 @@ impl TypeChecker { Some(import_use.clone()) } + /// Record a unique imported Rust trait method when receiver metadata is unavailable. + /// + /// rust-inspect can miss generated or re-export-heavy concrete types while still extracting the imported trait or + /// falling back to core extension-trait vocabulary. In that case the import itself is enough for Rust method + /// lookup; a recovered signature only adds call-site parameter shape metadata. + fn record_unique_rust_trait_import_for_unresolved_receiver_call( + &mut self, + method: &str, + span: Span, + ) -> Option { + let matches = self + .type_info + .rust + .trait_imports + .iter() + .filter(|(_, import)| import.methods.contains(method)) + .map(|(binding, import)| RustMethodTraitImportUse { + binding: binding.clone(), + trait_path: import.trait_path.clone(), + method: method.to_string(), + signature: Self::rust_trait_method_signature(import, method), + }) + .collect::>(); + let [import_use] = matches.as_slice() else { + return None; + }; + self.type_info + .record_rust_method_trait_import_use(span, import_use.clone()); + Some(import_use.clone()) + } + /// Return the trait method signature when `import` is implemented by `type_info` and declares `method`. fn rust_trait_import_matches_receiver( type_info: &incan_core::interop::RustTypeInfo, @@ -2382,6 +2489,9 @@ impl TypeChecker { } let resolve_on = |checker: &mut Self, ty: &ResolvedType| -> ResolvedType { + if field == "__name__" && checker.is_generic_placeholder_type(ty) { + return ResolvedType::Str; + } match ty { ResolvedType::Unknown => ResolvedType::Unknown, // Trait default methods typecheck against `Self`, but field access must be declared via @@ -2405,6 +2515,7 @@ impl TypeChecker { checker.errors.push(errors::missing_field(&ty.to_string(), field, span)); ResolvedType::Unknown } + ResolvedType::Function(_, _) if field == "__name__" => ResolvedType::Str, ResolvedType::Named(type_name) => { if let Some(field_ty) = checker.resolve_nominal_field_type(type_name, None, field, span) { return field_ty; @@ -2430,6 +2541,9 @@ impl TypeChecker { ResolvedType::Unknown } ResolvedType::TypeVar(name) => { + if field == "__name__" { + return ResolvedType::Str; + } if let Some(property_ty) = checker.resolve_generic_placeholder_property(name, field, span) { return property_ty; } @@ -2867,7 +2981,8 @@ impl TypeChecker { // Rust: `Option<&T>::copied() -> Option` (for `T: Copy`). if let ResolvedType::Ref(t) | ResolvedType::RefMut(t) = inner { let t = (*t).clone(); - if matches!(t, ResolvedType::Int | ResolvedType::Float | ResolvedType::Bool) { + let is_unresolved_rust_generic = matches!(&t, ResolvedType::RustPath(path) if TypeChecker::rust_display_type_var_name(path).is_some()); + if self.is_copy_type(&t) || self.is_generic_placeholder_type(&t) || is_unresolved_rust_generic { return option_ty(t); } } @@ -2891,6 +3006,42 @@ impl TypeChecker { } } + if let ResolvedType::Generic(name, type_args) = &base_ty + && collection_type_id(name.as_str()) == Some(CollectionTypeId::Result) + && type_args.len() == 2 + { + let ok_ty = type_args[0].clone(); + match result_methods::from_str(method) { + Some(result_methods::ResultMethodId::Unwrap) => { + if !args.is_empty() { + self.errors.push(errors::type_mismatch( + "no arguments", + &format!("{} argument(s)", args.len()), + span, + )); + } + return ok_ty; + } + Some(result_methods::ResultMethodId::UnwrapOr) => { + if let Some(default_ty) = arg_types.first() + && !self.types_compatible(default_ty, &ok_ty) + { + self.errors + .push(errors::type_mismatch(&ok_ty.to_string(), &default_ty.to_string(), span)); + } + if args.len() != 1 { + self.errors.push(errors::type_mismatch( + "one default argument", + &format!("{} argument(s)", args.len()), + span, + )); + } + return ok_ty; + } + _ => {} + } + } + if let ResolvedType::Generic(name, type_args) = &base_ty && collection_type_id(name.as_str()) == Some(CollectionTypeId::Result) && type_args.len() == 2 @@ -2910,20 +3061,21 @@ impl TypeChecker { if let ResolvedType::Generic(name, type_args) = &base_ty { if collection_type_id(name.as_str()) == Some(CollectionTypeId::Generator) { let elem = type_args.first().cloned().unwrap_or(ResolvedType::Unknown); - match method { - "map" => { + use iterator_methods::IteratorMethodId as M; + match iterator_methods::from_str(method) { + Some(M::Map) => { let mapped = self.generator_map_return_type(&elem, args, &arg_types, span); return generator_ty(mapped); } - "filter" => { + Some(M::Filter) => { self.validate_generator_filter_arg(&elem, args, &arg_types, span); return generator_ty(elem); } - "take" => { + Some(M::Take) => { self.validate_generator_take_arg(args, &arg_types, span); return generator_ty(elem); } - "collect" => { + Some(M::Collect) => { if !args.is_empty() { self.errors.push(errors::type_mismatch( "no arguments", @@ -3017,6 +3169,12 @@ impl TypeChecker { } } + if let Some(ret) = + self.resolve_union_clone_trait_method_call(&base_ty, method, type_args, args, &arg_types, span) + { + return ret; + } + if let ResolvedType::Generic(type_name, _type_args) = &base_ty && let Some(type_info) = self.lookup_semantic_type_info(type_name).cloned() { @@ -3263,6 +3421,37 @@ impl TypeChecker { ResolvedType::Unknown } + /// Resolve methods supplied by Clone for anonymous union wrappers. + fn resolve_union_clone_trait_method_call( + &mut self, + receiver_ty: &ResolvedType, + method: &str, + type_args: &[Spanned], + args: &[CallArg], + arg_types: &[ResolvedType], + span: Span, + ) -> Option { + if !receiver_ty.is_union() { + return None; + } + + let adoption = TypeBoundInfo { + name: core_traits::as_str(TraitId::Clone).to_string(), + source_name: None, + type_args: Vec::new(), + module_path: None, + }; + let method_info = self.trait_method_info_resolved_for_adoption(&adoption, method, span)?; + if !self.is_clone_type(receiver_ty) { + self.errors.push(CompileError::type_error( + format!("Union type '{receiver_ty}' cannot use '{method}(...)' because not all variants are cloneable"), + span, + )); + return Some(ResolvedType::Unknown); + } + Some(self.check_generic_method_call(method, method_info, type_args, args, arg_types, span, receiver_ty)) + } + /// Return known method result types for Rust imports when rust-inspect metadata is not specific enough. fn known_rust_path_method_return(path: &str, method: &str) -> Option { use incan_core::lang::types::numerics::NumericTypeId as N; diff --git a/src/frontend/typechecker/check_expr/calls/builtins.rs b/src/frontend/typechecker/check_expr/calls/builtins.rs index a2af26d76..f1870b73c 100644 --- a/src/frontend/typechecker/check_expr/calls/builtins.rs +++ b/src/frontend/typechecker/check_expr/calls/builtins.rs @@ -8,7 +8,7 @@ use crate::frontend::typechecker::helpers::{collection_type_id, dict_ty, list_ty use incan_core::lang::builtins::{self as core_builtins, BuiltinFnId}; use incan_core::lang::stdlib; use incan_core::lang::surface::constructors::{self as surface_constructors, ConstructorId}; -use incan_core::lang::surface::functions::{self as surface_functions, SurfaceFnId}; +use incan_core::lang::surface::functions::SurfaceFnId; use incan_core::lang::surface::types::{self as surface_types, SurfaceTypeId}; use incan_core::lang::types::collections::CollectionTypeId; @@ -118,10 +118,19 @@ impl TypeChecker { call_span: Span, respect_shadowing: bool, ) -> Option { - let has_function_symbol = respect_shadowing && self.has_non_builtin_function_definition(name); + let has_call_root_binding = respect_shadowing && self.has_non_builtin_call_root_binding(name); + let surface_function_binding = respect_shadowing + .then(|| self.active_surface_function_import(name)) + .flatten(); + let surface_type_binding = respect_shadowing + .then(|| self.active_surface_type_import(name)) + .flatten(); // Constructors (variant-like) if let Some(cid) = surface_constructors::from_str(name) { + if has_call_root_binding { + return None; + } return match cid { ConstructorId::Ok | ConstructorId::Err => { let arg_types = self.check_call_arg_types(args); @@ -178,7 +187,7 @@ impl TypeChecker { // Core builtin functions (registry-driven) if let Some(bid) = core_builtins::from_str(name) { - if has_function_symbol { + if has_call_root_binding { return None; } return match bid { @@ -459,10 +468,7 @@ impl TypeChecker { } // Surface/runtime functions (registry-driven) - if let Some(fid) = surface_functions::from_str(name) { - if !has_function_symbol { - return None; - } + if let Some(fid) = surface_function_binding { return match fid { SurfaceFnId::SleepMs => { if let Some(arg) = args.first() { @@ -542,7 +548,17 @@ impl TypeChecker { } // Surface types that behave like constructors and whose result type depends on args. - if let Some(tid) = surface_types::from_str(name) { + let surface_type = surface_type_binding.or_else(|| { + if has_call_root_binding { + None + } else { + surface_types::from_str(name) + } + }); + if let Some(tid) = surface_type { + if has_call_root_binding { + debug_assert_eq!(surface_type_binding, Some(tid)); + } return match tid { SurfaceTypeId::Json | SurfaceTypeId::Query => { Some(self.check_json_query_constructor_call(tid, args, call_span)) @@ -587,6 +603,9 @@ impl TypeChecker { // Python-like type conversion helpers (surface). These are not part of `lang::builtins`. if let Some(cid) = collection_type_id(name) { + if has_call_root_binding { + return None; + } return match cid { CollectionTypeId::Dict => { let (key_ty, val_ty) = if let Some(arg) = args.first() { diff --git a/src/frontend/typechecker/check_expr/calls/generic_bounds.rs b/src/frontend/typechecker/check_expr/calls/generic_bounds.rs index 54f899af5..eca2eaa11 100644 --- a/src/frontend/typechecker/check_expr/calls/generic_bounds.rs +++ b/src/frontend/typechecker/check_expr/calls/generic_bounds.rs @@ -5,13 +5,6 @@ use crate::frontend::ast::{CallArg, Span, Spanned, Type}; use crate::frontend::diagnostics::errors; use crate::frontend::resolved_type_subst::{substitute_resolved_type, type_param_subst_map_call_site}; use crate::frontend::symbols::{CallableParam, FunctionInfo, MethodInfo, ResolvedType, TypeInfo}; -use crate::frontend::typechecker::helpers::collection_type_id; -use incan_core::interop::is_rust_capability_bound; -use incan_core::lang::derives::{self, DeriveId}; -use incan_core::lang::trait_capabilities::{self, TraitCapabilityInfo, TraitCapabilityType}; -use incan_core::lang::traits::{self as builtin_traits, TraitId}; -use incan_core::lang::types::collections::CollectionTypeId; -use incan_core::lang::types::numerics; impl TypeChecker { /// Validate generic function call type arguments, value arguments, and explicit type-parameter bounds. @@ -397,649 +390,4 @@ impl TypeChecker { } } } - - /// Return the active generic placeholder name represented by `ty`. - /// - /// Function bodies sometimes resolve an in-scope type parameter as a named placeholder while validating a nested - /// generic call. Treat that spelling as a placeholder only when the current bound stack actually contains it. - fn active_type_param_name<'a>(&self, ty: &'a ResolvedType) -> Option<&'a str> { - let name = match ty { - ResolvedType::TypeVar(name) | ResolvedType::Named(name) => name, - _ => return None, - }; - self.current_type_param_bound_details - .iter() - .rev() - .any(|frame| frame.contains_key(name)) - .then_some(name.as_str()) - } - - /// Check whether an active generic placeholder already carries the bound required by a nested generic call. - fn active_type_param_satisfies_bound_info( - &self, - placeholder_name: &str, - required: &crate::frontend::symbols::TypeBoundInfo, - bindings: &std::collections::HashMap, - ) -> bool { - for frame in self.current_type_param_bound_details.iter().rev() { - let Some(active_bounds) = frame.get(placeholder_name) else { - continue; - }; - for active in active_bounds { - if !Self::type_bound_names_match(active, required) { - continue; - } - if required.type_args.is_empty() { - return true; - } - if active.type_args.len() != required.type_args.len() { - continue; - } - let expected = required - .type_args - .iter() - .map(|arg| substitute_resolved_type(arg, bindings)); - let actual = active - .type_args - .iter() - .map(|arg| substitute_resolved_type(arg, bindings)); - if expected - .zip(actual) - .all(|(left, right)| self.types_compatible(&left, &right)) - { - return true; - } - } - return false; - } - false - } - - /// Render a type-parameter bound with call-site substitutions applied. - fn type_bound_display( - &self, - bound: &crate::frontend::symbols::TypeBoundInfo, - bindings: &std::collections::HashMap, - ) -> String { - if bound.type_args.is_empty() { - return bound.name.clone(); - } - let args = bound - .type_args - .iter() - .map(|arg| substitute_resolved_type(arg, bindings).to_string()) - .collect::>() - .join(", "); - format!("{}[{}]", bound.name, args) - } - - /// Return the resolved source trait item name for a bound, falling back to the visible spelling. - fn type_bound_source_name(bound: &crate::frontend::symbols::TypeBoundInfo) -> &str { - bound - .source_name - .as_deref() - .unwrap_or_else(|| bound.name.rsplit('.').next().unwrap_or(bound.name.as_str())) - } - - /// Return whether two bound records identify the same trait, accounting for import aliases. - fn type_bound_names_match( - left: &crate::frontend::symbols::TypeBoundInfo, - right: &crate::frontend::symbols::TypeBoundInfo, - ) -> bool { - if left.name == right.name { - return true; - } - left.module_path == right.module_path - && left.module_path.is_some() - && Self::type_bound_source_name(left) == Self::type_bound_source_name(right) - } - - /// Return whether a type satisfies one explicit bound, including generic trait arguments. - pub(crate) fn type_satisfies_explicit_bound_info( - &self, - ty: &ResolvedType, - bound: &crate::frontend::symbols::TypeBoundInfo, - bindings: &std::collections::HashMap, - ) -> bool { - if let Some(placeholder_name) = self.active_type_param_name(ty) - && self.active_type_param_satisfies_bound_info(placeholder_name, bound, bindings) - { - return true; - } - if bound.name == builtin_traits::as_str(TraitId::Awaitable) { - let expected_output = bound - .type_args - .first() - .map(|arg| substitute_resolved_type(arg, bindings)); - return self.type_satisfies_awaitable_bound(ty, expected_output.as_ref()); - } - if let Some(capability) = self.temporary_trait_capability_for_bound_info(bound) - && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) - { - return satisfies; - } - if bound.type_args.is_empty() { - return self.type_satisfies_explicit_bound(ty, &bound.name); - } - if is_rust_capability_bound(&bound.name) { - return true; - } - if builtin_traits::from_str(&bound.name).is_some() || self.lookup_semantic_trait_info(&bound.name).is_none() { - return self.type_satisfies_explicit_bound(ty, &bound.name); - } - let expected_args = bound - .type_args - .iter() - .map(|arg| substitute_resolved_type(arg, bindings)) - .collect::>(); - self.type_satisfies_nominal_trait_bound_with_args(ty, &bound.name, &expected_args) - } - - /// Best-effort check whether a concrete type satisfies an explicit generic bound. - fn type_satisfies_explicit_bound(&self, ty: &ResolvedType, bound: &str) -> bool { - if bound == builtin_traits::as_str(TraitId::Awaitable) { - return self.type_satisfies_awaitable_bound(ty, None); - } - // `std.rust` markers (`Send`, `Sync`, …) are enforced when lowering to Rust, not here. - if is_rust_capability_bound(bound) { - return true; - } - if let Some(capability) = self.temporary_trait_capability_for_bound(bound) - && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) - { - return satisfies; - } - // For non-builtin traits, apply nominal trait/supertrait compatibility (RFC 042) directly. - // - // This keeps capability checks language-general and avoids ad hoc receiver-category gating. - if builtin_traits::from_str(bound).is_none() && self.lookup_semantic_trait_info(bound).is_some() { - return self.type_satisfies_nominal_trait_bound(ty, bound); - } - match ty { - // Unknown / still-generic types are kept permissive to avoid cascading errors. - ResolvedType::Unknown - | ResolvedType::TypeVar(_) - | ResolvedType::RustPath(_) - | ResolvedType::CallSiteInfer => true, - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Numeric(_) - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit => self.primitive_type_satisfies_bound(ty, bound), - ResolvedType::Tuple(items) => self.tuple_type_satisfies_bound(items, bound), - ResolvedType::FrozenList(inner) => self.collection_type_satisfies_bound( - CollectionTypeId::FrozenList, - std::slice::from_ref(inner.as_ref()), - bound, - ), - ResolvedType::FrozenSet(inner) => self.collection_type_satisfies_bound( - CollectionTypeId::FrozenSet, - std::slice::from_ref(inner.as_ref()), - bound, - ), - ResolvedType::FrozenDict(k, v) => { - let pair = [k.as_ref().clone(), v.as_ref().clone()]; - self.collection_type_satisfies_bound(CollectionTypeId::FrozenDict, &pair, bound) - } - ResolvedType::Generic(name, args) => { - if let Some(kind) = collection_type_id(name.as_str()) { - self.collection_type_satisfies_bound(kind, args, bound) - } else { - self.named_type_satisfies_bound(name, bound) - } - } - ResolvedType::Named(type_name) => self.named_type_satisfies_bound(type_name, bound), - ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => self.type_satisfies_explicit_bound(inner, bound), - ResolvedType::Function(_, _) | ResolvedType::SelfType => false, - } - } - - /// Check whether `ty` satisfies a nominal trait bound `bound_trait` under RFC 042 semantics. - /// - /// This path is used for non-builtin traits. It intentionally reuses existing trait compatibility helpers: - /// - concrete adopters satisfy direct and transitive supertraits via `type_implements_trait` - /// - trait-typed values satisfy broader traits via `trait_is_supertrait_of` - fn type_satisfies_nominal_trait_bound(&self, ty: &ResolvedType, bound_trait: &str) -> bool { - match ty { - // Keep unknown / generic placeholders permissive to avoid cascading diagnostics. - ResolvedType::Unknown - | ResolvedType::TypeVar(_) - | ResolvedType::RustPath(_) - | ResolvedType::CallSiteInfer => true, - ResolvedType::Named(type_name) => { - if self.lookup_semantic_trait_info(type_name).is_some() { - self.trait_is_supertrait_of(type_name, bound_trait) - } else { - self.type_implements_trait(type_name, bound_trait) - } - } - ResolvedType::Generic(type_name, _args) => { - if self.lookup_semantic_trait_info(type_name).is_some() { - self.trait_is_supertrait_of(type_name, bound_trait) - } else if self.lookup_semantic_type_info(type_name).is_some() { - self.type_implements_trait(type_name, bound_trait) - } else { - false - } - } - ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { - self.type_satisfies_nominal_trait_bound(inner, bound_trait) - } - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Numeric(_) - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - | ResolvedType::Tuple(_) - | ResolvedType::FrozenList(_) - | ResolvedType::FrozenSet(_) - | ResolvedType::FrozenDict(_, _) - | ResolvedType::Function(_, _) - | ResolvedType::SelfType => false, - } - } - - /// Return whether a nominal type satisfies a trait bound with exact expected trait arguments. - fn type_satisfies_nominal_trait_bound_with_args( - &self, - ty: &ResolvedType, - bound_trait: &str, - expected_args: &[ResolvedType], - ) -> bool { - match ty { - ResolvedType::Unknown - | ResolvedType::TypeVar(_) - | ResolvedType::RustPath(_) - | ResolvedType::CallSiteInfer => true, - ResolvedType::Named(type_name) => { - self.type_implements_trait_with_args(type_name, &[], bound_trait, expected_args) - } - ResolvedType::Generic(type_name, type_args) => { - self.type_implements_trait_with_args(type_name, type_args, bound_trait, expected_args) - } - ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { - self.type_satisfies_nominal_trait_bound_with_args(inner, bound_trait, expected_args) - } - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Numeric(_) - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - | ResolvedType::Tuple(_) - | ResolvedType::FrozenList(_) - | ResolvedType::FrozenSet(_) - | ResolvedType::FrozenDict(_, _) - | ResolvedType::Function(_, _) - | ResolvedType::SelfType => false, - } - } - - /// Check a concrete model/class adoption list for a matching generic trait instantiation. - fn type_implements_trait_with_args( - &self, - type_name: &str, - concrete_type_args: &[ResolvedType], - bound_trait: &str, - expected_args: &[ResolvedType], - ) -> bool { - let Some(info) = self.lookup_semantic_type_info(type_name) else { - return false; - }; - let (owner_type_params, adoptions, derives) = match info { - TypeInfo::Model(model) => ( - model.type_params.as_slice(), - model.trait_adoptions.as_slice(), - Some(model.derives.as_slice()), - ), - TypeInfo::Class(class) => ( - class.type_params.as_slice(), - class.trait_adoptions.as_slice(), - Some(class.derives.as_slice()), - ), - TypeInfo::Enum(en) => ( - en.type_params.as_slice(), - en.trait_adoptions.as_slice(), - Some(en.derives.as_slice()), - ), - TypeInfo::Newtype(newtype) => (newtype.type_params.as_slice(), newtype.trait_adoptions.as_slice(), None), - TypeInfo::Builtin | TypeInfo::TypeAlias => return false, - }; - - if expected_args.is_empty() - && derives.is_some_and(|items| items.iter().any(|derive| derive == bound_trait)) - && self.lookup_semantic_trait_info(bound_trait).is_some() - { - return true; - } - - let owner_subst = - crate::frontend::resolved_type_subst::type_param_subst_map(owner_type_params, concrete_type_args); - for adoption in adoptions { - let Some(adopted_info) = self.lookup_semantic_trait_info(&adoption.name) else { - continue; - }; - let direct_args = if adoption.type_args.is_empty() { - concrete_type_args - .iter() - .take(adopted_info.type_params.len()) - .cloned() - .collect::>() - } else { - adoption - .type_args - .iter() - .map(|arg| substitute_resolved_type(arg, &owner_subst)) - .collect::>() - }; - if direct_args.len() != adopted_info.type_params.len() { - continue; - } - if self.trait_name_matches(&adoption.name, bound_trait) - && self.trait_args_match(&direct_args, expected_args) - { - return true; - } - - let subst = - crate::frontend::resolved_type_subst::type_param_subst_map(&adopted_info.type_params, &direct_args); - for (supertrait_name, supertrait_args) in self.semantic_supertrait_closure(&adoption.name) { - if !self.trait_name_matches(&supertrait_name, bound_trait) { - continue; - } - let instantiated = supertrait_args - .iter() - .map(|arg| substitute_resolved_type(arg, &subst)) - .collect::>(); - if self.trait_args_match(&instantiated, expected_args) { - return true; - } - } - } - false - } - - /// Compare instantiated trait arguments using the typechecker's compatibility relation. - fn trait_args_match(&self, actual_args: &[ResolvedType], expected_args: &[ResolvedType]) -> bool { - actual_args.len() == expected_args.len() - && actual_args - .iter() - .zip(expected_args.iter()) - .all(|(actual, expected)| self.types_compatible(actual, expected)) - } - - /// Return whether a primitive type satisfies a builtin or registry-backed temporary capability bound. - fn primitive_type_satisfies_bound(&self, ty: &ResolvedType, bound: &str) -> bool { - if bound == derives::as_str(DeriveId::Copy) { - return self.is_copy_type(ty); - } - if let Some(capability) = self.temporary_trait_capability_for_bound(bound) - && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) - { - return satisfies; - } - - match builtin_traits::from_str(bound) { - Some(TraitId::Clone | TraitId::Debug | TraitId::Display) => matches!( - ty, - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - ), - Some(TraitId::Default) => matches!( - ty, - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - ), - Some(TraitId::Awaitable) => self.type_satisfies_awaitable_bound(ty, None), - Some(TraitId::Eq | TraitId::Ord | TraitId::Hash) => matches!( - ty, - ResolvedType::Int - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - ), - Some(TraitId::PartialEq | TraitId::PartialOrd) => matches!( - ty, - ResolvedType::Int - | ResolvedType::Float - | ResolvedType::Bool - | ResolvedType::Str - | ResolvedType::Bytes - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - ), - _ => false, - } - } - - /// Resolve a temporary trait-owned capability bridge for a bound. - /// - /// This keeps RFC 101's v0.3 bridge explicit until RFC 098/099 can express the same conformance family in source. - fn temporary_trait_capability_for_bound(&self, bound: &str) -> Option<&'static TraitCapabilityInfo> { - let (module_path, trait_name) = self.resolve_bound_trait_path(bound)?; - let capability = trait_capabilities::for_trait_path(&module_path, &trait_name)?; - let info = self - .lookup_semantic_trait_info(bound) - .or_else(|| self.lookup_semantic_trait_info(capability.trait_name))?; - capability - .required_methods - .iter() - .all(|method| info.methods.contains_key(*method)) - .then_some(capability) - } - - /// Resolve a temporary capability bridge from a checked bound that may have crossed a package manifest boundary. - fn temporary_trait_capability_for_bound_info( - &self, - bound: &crate::frontend::symbols::TypeBoundInfo, - ) -> Option<&'static TraitCapabilityInfo> { - if let Some(module_path) = &bound.module_path { - let trait_name = Self::type_bound_source_name(bound); - return trait_capabilities::for_trait_path(module_path, trait_name); - } - self.temporary_trait_capability_for_bound(&bound.name) - } - - /// Resolve a bound spelling to its defining module path and trait name. - fn resolve_bound_trait_path(&self, bound: &str) -> Option<(Vec, String)> { - if let Some(path) = self.import_aliases.get(bound) - && path.len() >= 2 - { - let trait_name = path.last()?.clone(); - let module_path = path[..path.len() - 1].to_vec(); - return Some((module_path, trait_name)); - } - if !bound.contains('.') { - let module_path = self.current_module_path.clone()?; - return Some((module_path, bound.to_string())); - } - let (module_name, trait_name) = bound.rsplit_once('.')?; - let module_path = self.module_path_for_imported_name(module_name)?; - Some((module_path, trait_name.to_string())) - } - - /// Return temporary trait satisfaction for proven source type families. - /// - /// Unresolved shapes stay permissive so ordinary inference and Rust interop can finish before a later concrete - /// substitution proves or rejects the capability. `None` means this bridge has no opinion and nominal lookup should - /// continue. - fn temporary_trait_capability_supports_type( - &self, - capability: &TraitCapabilityInfo, - ty: &ResolvedType, - ) -> Option { - match ty { - ResolvedType::Unknown - | ResolvedType::TypeVar(_) - | ResolvedType::RustPath(_) - | ResolvedType::CallSiteInfer => Some(true), - ResolvedType::Int => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Int)), - ResolvedType::Bool => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Bool)), - ResolvedType::Str => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Str)), - ResolvedType::Bytes => Some(trait_capabilities::supports_type( - capability, - TraitCapabilityType::Bytes, - )), - ResolvedType::Numeric(id) => Some(trait_capabilities::supports_type( - capability, - TraitCapabilityType::Numeric(*id), - )), - ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { - self.temporary_trait_capability_supports_type(capability, inner) - } - ResolvedType::Generic(name, args) - if numerics::decimal_constructor_from_str(name.as_str()).is_some() - && args.len() == 2 - && args - .iter() - .all(|arg| matches!(arg, ResolvedType::TypeVar(value) if value.parse::().is_ok())) => - { - Some(trait_capabilities::supports_type( - capability, - TraitCapabilityType::Decimal, - )) - } - ResolvedType::Named(type_name) | ResolvedType::Generic(type_name, _) - if self.value_enum_type_satisfies_temporary_trait_capability(type_name) => - { - Some(trait_capabilities::supports_type( - capability, - TraitCapabilityType::ValueEnum, - )) - } - ResolvedType::Float - | ResolvedType::FrozenStr - | ResolvedType::FrozenBytes - | ResolvedType::Unit - | ResolvedType::Tuple(_) - | ResolvedType::FrozenList(_) - | ResolvedType::FrozenSet(_) - | ResolvedType::FrozenDict(_, _) - | ResolvedType::Function(_, _) - | ResolvedType::SelfType => Some(false), - ResolvedType::Generic(_, _) | ResolvedType::Named(_) => None, - } - } - - /// Return whether a nominal type is a stable scalar value enum category for temporary capability bridges. - fn value_enum_type_satisfies_temporary_trait_capability(&self, type_name: &str) -> bool { - matches!( - self.lookup_semantic_type_info(type_name), - Some(crate::frontend::symbols::TypeInfo::Enum(info)) if info.value_enum.is_some() - ) - } - - fn tuple_type_satisfies_bound(&self, items: &[ResolvedType], bound: &str) -> bool { - match builtin_traits::from_str(bound) { - Some( - TraitId::Clone - | TraitId::Debug - | TraitId::Default - | TraitId::Eq - | TraitId::PartialEq - | TraitId::Ord - | TraitId::PartialOrd - | TraitId::Hash, - ) => items.iter().all(|item| self.type_satisfies_explicit_bound(item, bound)), - _ => false, - } - } - - fn collection_type_satisfies_bound(&self, kind: CollectionTypeId, args: &[ResolvedType], bound: &str) -> bool { - let all_args_satisfy = || args.iter().all(|arg| self.type_satisfies_explicit_bound(arg, bound)); - match builtin_traits::from_str(bound) { - Some(TraitId::Clone | TraitId::Debug) => all_args_satisfy(), - Some(TraitId::Default) => matches!( - kind, - CollectionTypeId::List - | CollectionTypeId::FrozenList - | CollectionTypeId::Dict - | CollectionTypeId::FrozenDict - | CollectionTypeId::Set - | CollectionTypeId::FrozenSet - | CollectionTypeId::Option - ), - Some(TraitId::Eq | TraitId::PartialEq) => all_args_satisfy(), - Some(TraitId::Ord | TraitId::PartialOrd) => { - matches!( - kind, - CollectionTypeId::List - | CollectionTypeId::FrozenList - | CollectionTypeId::Tuple - | CollectionTypeId::Option - ) && all_args_satisfy() - } - Some(TraitId::Hash) => { - matches!( - kind, - CollectionTypeId::List - | CollectionTypeId::FrozenList - | CollectionTypeId::Tuple - | CollectionTypeId::Option - ) && all_args_satisfy() - } - _ => false, - } - } - - /// Return whether `ty` is one of the checked await-realization paths for `Awaitable[T]`. - fn type_satisfies_awaitable_bound(&self, ty: &ResolvedType, expected_output: Option<&ResolvedType>) -> bool { - let Some(output_ty) = self.awaitable_output_type_for_known_type(ty) else { - return false; - }; - expected_output.is_none_or(|expected| { - matches!(output_ty, ResolvedType::Unknown) || self.types_compatible(&output_ty, expected) - }) - } - - /// Resolve the output type for known awaitable carrier types. - fn awaitable_output_type_for_known_type(&self, ty: &ResolvedType) -> Option { - self.await_output_type_from_type(ty) - } - - /// Return whether a named user type explicitly satisfies a generic trait bound. - fn named_type_satisfies_bound(&self, type_name: &str, bound: &str) -> bool { - match self.lookup_type_info(type_name) { - Some(TypeInfo::Builtin) => matches!(builtin_traits::from_str(bound), Some(TraitId::Clone | TraitId::Debug)), - Some(TypeInfo::Model(info)) => { - info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) - } - Some(TypeInfo::Class(info)) => { - info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) - } - Some(TypeInfo::Enum(info)) => { - info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) - } - Some(TypeInfo::Newtype(info)) => info.traits.iter().any(|t| t == bound), - Some(TypeInfo::TypeAlias) => false, - None => false, - } - } } diff --git a/src/frontend/typechecker/check_expr/calls/rust_boundary.rs b/src/frontend/typechecker/check_expr/calls/rust_boundary.rs index e54bd61a7..c7b134e31 100644 --- a/src/frontend/typechecker/check_expr/calls/rust_boundary.rs +++ b/src/frontend/typechecker/check_expr/calls/rust_boundary.rs @@ -268,20 +268,22 @@ impl TypeChecker { /// This first tries builtin coercion-matrix matches, then resolved-type compatibility, then rusttype-specific /// boundary adapters. fn rust_arg_boundary_match(&self, arg_ty: &ResolvedType, rust_param_ty: &str) -> RustArgBoundaryMatch { - let normalized = rust_param_ty.replace(' ', ""); + let display = Self::rust_display_without_lifetimes(rust_param_ty); + let normalized = display.replace(' ', ""); if Self::rust_display_type_var_name(normalized.as_str()).is_some() { return RustArgBoundaryMatch::Exact; } - let borrowed_shared = matches!(Self::rust_display_borrow_kind(normalized.as_str()), Some((false, _))); - if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(normalized.as_str()) { - if Self::is_rust_generic_type_param_display(inner) + let borrowed_shared = matches!(Self::rust_display_borrow_kind(display.as_str()), Some((false, _))); + if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(display.as_str()) { + let inner_normalized = Self::compact_rust_display(inner); + if Self::is_rust_generic_type_param_display(inner_normalized.as_str()) && !is_mut && !matches!(arg_ty, ResolvedType::Ref(_) | ResolvedType::RefMut(_)) { return RustArgBoundaryMatch::Exact; } if !is_mut { - let target_inner_ty = self.resolved_type_from_rust_display(inner); + let target_inner_ty = self.resolved_type_from_rust_display(inner_normalized.as_str()); if Self::incan_boundary_type_display(arg_ty).is_none() && self.types_compatible(arg_ty, &target_inner_ty) { @@ -289,13 +291,13 @@ impl TypeChecker { } } if is_mut { - let target_inner_ty = self.resolved_type_from_rust_display(inner); + let target_inner_ty = self.resolved_type_from_rust_display(inner_normalized.as_str()); if self.types_compatible(arg_ty, &target_inner_ty) { return RustArgBoundaryMatch::Exact; } if let Some(incan_display) = Self::incan_boundary_type_display(arg_ty) && let Some(CoercionPolicy::Exact) = - admitted_builtin_coercion(incan_display.as_str(), inner.replace(' ', "").as_str()) + admitted_builtin_coercion(incan_display.as_str(), inner_normalized.as_str()) { return RustArgBoundaryMatch::Exact; } @@ -331,7 +333,9 @@ impl TypeChecker { let params: Vec = params .iter() .map(|param| { - CallableParam::positional(self.resolved_param_type_from_rust_display(param.type_display.as_str())) + CallableParam::positional( + self.resolved_rust_boundary_target_from_param_display(param.type_display.as_str()), + ) }) .collect(); // Plain Rust type variables carry by-value shape, but they are not ordinary borrow-boundary snapshots. @@ -409,7 +413,7 @@ impl TypeChecker { for ((arg, arg_ty), param) in args.iter().zip(arg_types.iter()).zip(params.iter()) { let arg_expr = Self::call_arg_expr(arg); let normalized = param.type_display.replace(' ', ""); - let target_ty = self.resolved_type_from_rust_display(normalized.as_str()); + let target_ty = self.resolved_rust_boundary_target_from_param_display(param.type_display.as_str()); if preserves_lookup_arg_shape && self.rust_lookup_probe_boundary_match(arg_ty, &target_ty) { continue; } @@ -462,7 +466,7 @@ impl TypeChecker { for ((arg, arg_ty), param) in args.iter().zip(arg_types.iter()).zip(sig.params.iter()) { let arg_expr = Self::call_arg_expr(arg); let normalized = param.type_display.replace(' ', ""); - let target_ty = self.resolved_type_from_rust_display(normalized.as_str()); + let target_ty = self.resolved_rust_boundary_target_from_param_display(param.type_display.as_str()); match self.rust_arg_boundary_match(arg_ty, param.type_display.as_str()) { RustArgBoundaryMatch::Exact => {} RustArgBoundaryMatch::Coercion(kind) => { @@ -837,6 +841,89 @@ mod validate_rust_function_call_tests { .contains_key(&(span.start, span.end)), "expected rust arg coercion metadata for borrowed String boundary" ); + let expected = ResolvedType::Ref(Box::new(ResolvedType::RustPath("String".to_string()))); + assert_eq!( + checker + .type_info + .rust + .arg_coercions + .get(&(span.start, span.end)) + .map(|coercion| &coercion.target_type), + Some(&expected), + "borrowed owned Rust params must preserve owned target shape in lowering metadata" + ); + } + + #[test] + fn rust_function_call_accepts_string_for_borrowed_str_param() { + let mut checker = TypeChecker::new(); + let span = Span::new(10, 20); + let arg_expr = Spanned::new(Expr::Literal(Literal::String("{}".to_string())), span); + let args = [CallArg::Positional(arg_expr)]; + let sig = RustFunctionSig { + params: vec![RustParam { + name: Some("value".to_string()), + type_display: "&str".to_string(), + }], + return_type: "()".to_string(), + is_async: false, + is_unsafe: false, + }; + + let _ = checker.validate_rust_function_call("rust::demo::takes_borrowed_str", &sig, &args, span); + + assert!( + checker.errors.is_empty(), + "expected borrowed str boundary to admit Incan str, errors={:?}", + checker.errors + ); + let expected = ResolvedType::Ref(Box::new(ResolvedType::Str)); + assert_eq!( + checker + .type_info + .rust + .arg_coercions + .get(&(span.start, span.end)) + .map(|coercion| &coercion.target_type), + Some(&expected), + "borrowed str params must stay distinct from borrowed owned String params" + ); + } + + #[test] + fn rust_function_call_accepts_bytes_for_borrowed_vec_param() { + let mut checker = TypeChecker::new(); + let span = Span::new(10, 20); + let arg_expr = Spanned::new(Expr::Literal(Literal::Bytes(b"abc".to_vec())), span); + let args = [CallArg::Positional(arg_expr)]; + let sig = RustFunctionSig { + params: vec![RustParam { + name: Some("value".to_string()), + type_display: "&Vec".to_string(), + }], + return_type: "()".to_string(), + is_async: false, + is_unsafe: false, + }; + + let _ = checker.validate_rust_function_call("rust::demo::takes_borrowed_vec", &sig, &args, span); + + assert!( + checker.errors.is_empty(), + "expected borrowed Vec boundary to admit Incan bytes, errors={:?}", + checker.errors + ); + let expected = ResolvedType::Ref(Box::new(ResolvedType::RustPath("Vec".to_string()))); + assert_eq!( + checker + .type_info + .rust + .arg_coercions + .get(&(span.start, span.end)) + .map(|coercion| &coercion.target_type), + Some(&expected), + "borrowed owned Rust byte-vector params must preserve owned target shape in lowering metadata" + ); } #[test] @@ -999,6 +1086,8 @@ mod validate_rust_function_call_tests { let checker = TypeChecker::new(); assert!(checker.rust_arg_matches_boundary(&ResolvedType::Named("Payload".to_string()), "T",)); + assert!(checker.rust_arg_matches_boundary(&ResolvedType::Bytes, "impl Buf")); + assert!(checker.rust_arg_matches_boundary(&ResolvedType::Bytes, "implBuf")); assert!(checker.rust_arg_matches_boundary(&ResolvedType::Named("Payload".to_string()), "&T",)); assert!(checker.rust_arg_matches_boundary(&ResolvedType::Named("Payload".to_string()), "&TValue",)); } @@ -1046,6 +1135,54 @@ mod validate_rust_function_call_tests { ); } + #[test] + fn validate_rust_method_call_records_by_value_impl_trait_param_shape() { + let mut checker = TypeChecker::new(); + let span = Span::new(30, 40); + let arg_expr = Spanned::new(Expr::Ident("encoded".to_string()), span); + let args = [CallArg::Positional(arg_expr)]; + let arg_types = [ResolvedType::Bytes]; + let sig = RustFunctionSig { + params: vec![RustParam { + name: Some("buf".to_string()), + type_display: "implBuf".to_string(), + }], + return_type: "demo::FileDescriptorSet".to_string(), + is_async: false, + is_unsafe: false, + }; + + let _ = checker.validate_rust_method_call( + "rust::demo::FileDescriptorSet.decode", + &sig, + &args, + &arg_types, + false, + span, + ); + + assert!( + checker.errors.is_empty(), + "expected by-value impl Trait Rust param to accept bytes without borrow coercion, got {:?}", + checker.errors + ); + assert!( + checker.type_info.rust.arg_coercions.is_empty(), + "expected by-value impl Trait Rust param to avoid borrow coercion, got {:?}", + checker.type_info.rust.arg_coercions + ); + assert!( + checker + .type_info + .calls + .call_site_callable_params + .get(&(span.start, span.end)) + .is_some_and(|params| params.len() == 1 && params[0].ty == ResolvedType::TypeVar("implBuf".to_string())), + "expected Rust by-value impl Trait method param shape to be recorded, got {:?}", + checker.type_info.calls.call_site_callable_params + ); + } + #[test] fn validate_rust_method_call_records_interop_coercion_for_rusttype_target() { let mut checker = TypeChecker::new(); diff --git a/src/frontend/typechecker/check_expr/comps.rs b/src/frontend/typechecker/check_expr/comps.rs index 8dcd294a3..b702b65d9 100644 --- a/src/frontend/typechecker/check_expr/comps.rs +++ b/src/frontend/typechecker/check_expr/comps.rs @@ -4,6 +4,7 @@ //! and type-checking the generated element/value expressions in a nested scope. use crate::frontend::ast::*; +use crate::frontend::diagnostics::errors; use crate::frontend::symbols::*; use crate::frontend::typechecker::helpers::{dict_ty, generator_ty, list_ty}; @@ -94,6 +95,7 @@ impl TypeChecker { let prev_in_async_body = self.in_async_body; self.in_async_body = false; + let prev_return_error_type = self.current_return_error_type.take(); let param_types: Vec<_> = params .iter() @@ -114,9 +116,70 @@ impl TypeChecker { .collect(); let return_ty = self.check_expr(body); + self.current_return_error_type = prev_return_error_type; self.in_async_body = prev_in_async_body; self.symbols.exit_scope(); ResolvedType::Function(param_types, Box::new(return_ty)) } + + /// Type-check a closure expression against an expected function shape. + pub(in crate::frontend::typechecker::check_expr) fn check_closure_with_expected( + &mut self, + params: &[Spanned], + body: &Spanned, + expected_params: &[CallableParam], + expected_ret: &ResolvedType, + span: Span, + ) -> ResolvedType { + if params.len() != expected_params.len() { + self.errors.push(errors::builtin_arity( + "closure", + expected_params.len(), + params.len(), + span, + )); + return ResolvedType::Unknown; + } + + self.symbols.enter_scope(ScopeKind::Function); + + let prev_in_async_body = self.in_async_body; + self.in_async_body = false; + let prev_return_error_type = self.current_return_error_type.take(); + + let param_types: Vec<_> = params + .iter() + .zip(expected_params.iter()) + .map(|(param, expected)| { + let ty = expected.ty.clone(); + self.symbols.define(Symbol { + name: param.node.name.clone(), + kind: SymbolKind::Variable(VariableInfo { + ty: ty.clone(), + is_mutable: false, + is_used: false, + }), + span: param.span, + scope: 0, + }); + CallableParam::named(param.node.name.clone(), ty, param.node.kind) + }) + .collect(); + + let return_ty = self.check_expr_with_expected(body, Some(expected_ret)); + if !matches!(return_ty, ResolvedType::Unknown) && !self.types_compatible(&return_ty, expected_ret) { + self.errors.push(errors::type_mismatch( + &expected_ret.to_string(), + &return_ty.to_string(), + body.span, + )); + } + + self.current_return_error_type = prev_return_error_type; + self.in_async_body = prev_in_async_body; + self.symbols.exit_scope(); + + ResolvedType::Function(param_types, Box::new(expected_ret.clone())) + } } diff --git a/src/frontend/typechecker/check_expr/control_flow.rs b/src/frontend/typechecker/check_expr/control_flow.rs index 48f7028b7..fc94b7730 100644 --- a/src/frontend/typechecker/check_expr/control_flow.rs +++ b/src/frontend/typechecker/check_expr/control_flow.rs @@ -311,14 +311,16 @@ impl TypeChecker { return ResolvedType::Unknown; } - if let (Some(inner_err), Some(expected_err)) = (inner_ty.result_err_type(), &self.current_return_error_type) - && !self.types_compatible(inner_err, expected_err) - { - self.errors.push(errors::incompatible_error_type( - &expected_err.to_string(), - &inner_err.to_string(), - span, - )); + match (inner_ty.result_err_type(), self.current_return_error_type.clone()) { + (Some(inner_err), Some(expected_err)) if !self.types_compatible(inner_err, &expected_err) => { + self.errors.push(errors::incompatible_error_type( + &expected_err.to_string(), + &inner_err.to_string(), + span, + )); + } + (Some(_), None) => self.errors.push(errors::try_without_result_return(span)), + _ => {} } inner_ty.result_ok_type().cloned().unwrap_or(ResolvedType::Unknown) diff --git a/src/frontend/typechecker/check_expr/mod.rs b/src/frontend/typechecker/check_expr/mod.rs index 90036842d..e47f65ec4 100644 --- a/src/frontend/typechecker/check_expr/mod.rs +++ b/src/frontend/typechecker/check_expr/mod.rs @@ -185,8 +185,8 @@ impl TypeChecker { Expr::Constructor(name, args) => self.check_constructor(name, args, expr.span), Expr::FString(parts) => { for part in parts { - if let FStringPart::Expr(e) = part { - self.check_expr(e); + if let FStringPart::Expr { expr, .. } = part { + self.check_expr(expr); } } ResolvedType::Str @@ -283,6 +283,9 @@ impl TypeChecker { (Expr::MethodCall(base, method, type_args, args), Some(expected_ty)) => { self.check_method_call_with_expected(base, method, type_args, args, expr.span, Some(expected_ty)) } + (Expr::Closure(params, body), Some(ResolvedType::Function(expected_params, expected_ret))) => { + self.check_closure_with_expected(params, body, expected_params, expected_ret, expr.span) + } (Expr::List(elems), expected_ty) => self.check_list_with_expected(elems, expected_ty), (Expr::Dict(entries), expected_ty) => self.check_dict_with_expected(entries, expected_ty), (Expr::Loop(loop_expr), expected_ty) => self.check_loop_expr(loop_expr, expected_ty, expr.span), diff --git a/src/frontend/typechecker/collect.rs b/src/frontend/typechecker/collect.rs index ac71acdfe..d2f3eb541 100644 --- a/src/frontend/typechecker/collect.rs +++ b/src/frontend/typechecker/collect.rs @@ -10,7 +10,7 @@ use crate::frontend::symbols::*; use crate::frontend::typechecker::helpers::freeze_const_type; use incan_core::lang::decorators::{self as core_decorators, DecoratorId}; -use super::TypeChecker; +use super::{FunctionBindingInfo, TypeChecker}; mod decl_helpers; pub(super) mod decorators; @@ -1255,6 +1255,13 @@ impl TypeChecker { }) .collect(); let return_type = self.resolve_type_checked(&func.return_type); + self.type_info.declarations.function_bindings.insert( + func.name.clone(), + FunctionBindingInfo { + params: params.clone(), + return_type: return_type.clone(), + }, + ); self.symbols.define(Symbol { name: func.name.clone(), diff --git a/src/frontend/typechecker/collect/stdlib_imports.rs b/src/frontend/typechecker/collect/stdlib_imports.rs index 624973ea3..5cb4cd7d1 100644 --- a/src/frontend/typechecker/collect/stdlib_imports.rs +++ b/src/frontend/typechecker/collect/stdlib_imports.rs @@ -16,11 +16,12 @@ use crate::frontend::typechecker::type_info::RustTraitImportInfo; use crate::library_manifest::{ AliasExport, ClassExport, ConstExport, EnumExport, EnumValueExport, EnumValueTypeExport, FieldExport, FunctionExport, LibraryManifest, MethodExport, ModelExport, NewtypeExport, ParamExport, ParamKindExport, - PartialExport, ReceiverExport, StaticExport, TraitExport, TypeBoundExport, TypeParamExport, + PartialExport, ReceiverExport, StaticExport, TraitExport, TypeAliasExport, TypeBoundExport, TypeParamExport, resolved_type_from_manifest_type_ref, }; -use incan_core::interop::{RustItemKind, RustTraitAssoc, is_rust_capability_bound}; +use incan_core::interop::{RustItemKind, RustTraitAssoc, fallback_rust_trait_methods, is_rust_capability_bound}; use incan_core::lang::stdlib::{self, is_typechecker_only_stdlib}; +use incan_core::lang::surface::functions as surface_functions; use incan_core::lang::surface::types as surface_types; use incan_semantics_core::{DecoratorFeature, SurfaceFeatureKey}; @@ -36,7 +37,7 @@ enum ManifestExportRef<'a> { enum_name: &'a str, fields: &'a [crate::library_manifest::TypeRef], }, - TypeAlias, + TypeAlias(&'a TypeAliasExport), Newtype(&'a NewtypeExport), Const(&'a ConstExport), Static(&'a StaticExport), @@ -122,16 +123,12 @@ impl StdlibFromImportContext { }) } - /// Return `true` when a surface type import is legal from this stdlib module. - fn allows_surface_type_import(&self, item_name: &str) -> bool { - let Some(id) = surface_types::from_str(item_name) else { - return false; - }; - let Some(expected_module_path) = surface_types::stdlib_module_path(id) else { - return false; - }; + /// Return the imported surface type when it is legal from this stdlib module. + fn allowed_surface_type_import(&self, item_name: &str) -> Option { + let id = surface_types::from_str(item_name)?; + let expected_module_path = surface_types::stdlib_module_path(id)?; - match expected_module_path { + let allowed = match expected_module_path { "std.web" => self.is_web_namespace, "std.reflection" => self.is_reflection_module, _ if expected_module_path.starts_with("std.async.") => { @@ -140,7 +137,8 @@ impl StdlibFromImportContext { self.is_async_namespace && (async_root_or_prelude || self.module_path_str == expected_module_path) } _ => false, - } + }; + allowed.then_some(id) } } @@ -384,8 +382,12 @@ impl TypeChecker { if self.materialize_typechecker_only_stdlib_import(context.module, item, span) { return true; } - if stdlib_context.allows_surface_type_import(&item.name) { - self.define_from_import_symbol(item, SymbolKind::Type(TypeInfo::Builtin), span); + if let Some(surface_type) = stdlib_context.allowed_surface_type_import(&item.name) { + let local_name = Self::import_item_local_name(item); + let symbol_id = + self.define_named_import_symbol(local_name.clone(), SymbolKind::Type(TypeInfo::Builtin), span); + self.surface_type_import_bindings + .insert(local_name, (surface_type, symbol_id)); return true; } if self.materialize_stdlib_submodule_import(context.module, item, span) { @@ -451,8 +453,13 @@ impl TypeChecker { ) -> bool { if let Some(info) = self.stdlib_cache.lookup_function(&context.module.segments, &item.name) { let local_name = Self::import_item_local_name(item); + let surface_function = surface_functions::from_str(&item.name); self.record_testing_marker_import(context, item, &local_name, testing_semantics); - self.define_named_import_symbol(local_name, SymbolKind::Function(info), span); + let symbol_id = self.define_named_import_symbol(local_name.clone(), SymbolKind::Function(info), span); + if let Some(surface_function) = surface_function { + self.surface_function_import_bindings + .insert(local_name, (surface_function, symbol_id)); + } return true; } @@ -565,14 +572,14 @@ impl TypeChecker { } /// Define one already named imported symbol after root namespace validation. - fn define_named_import_symbol(&mut self, name: Ident, kind: SymbolKind, span: Span) { + fn define_named_import_symbol(&mut self, name: Ident, kind: SymbolKind, span: Span) -> SymbolId { self.validate_root_namespace(&name, span); self.symbols.define(Symbol { name, kind, span, scope: 0, - }); + }) } fn validate_pub_library_entry(&mut self, library: &str, span: Span) { @@ -713,7 +720,7 @@ impl TypeChecker { ManifestExportRef::Partial(export) => SymbolKind::Function(self.partial_info_from_manifest(export)), ManifestExportRef::Trait(export) => SymbolKind::Trait(self.trait_info_from_manifest(export)), ManifestExportRef::Enum(export) => SymbolKind::Type(TypeInfo::Enum(self.enum_info_from_manifest(export))), - ManifestExportRef::TypeAlias => SymbolKind::Type(TypeInfo::TypeAlias), + ManifestExportRef::TypeAlias(_) => SymbolKind::Type(TypeInfo::TypeAlias), ManifestExportRef::Newtype(export) => { SymbolKind::Type(TypeInfo::Newtype(self.newtype_info_from_manifest(export))) } @@ -733,6 +740,9 @@ impl TypeChecker { fields: fields.iter().map(resolved_type_from_manifest_type_ref).collect(), }), ManifestExportRef::Alias(export) => { + if let Some(function) = &export.projected_function { + return Some(SymbolKind::Function(self.function_info_from_manifest(function))); + } let target_name = export.target_path.last()?; return self.lookup_pub_library_symbol_member(library, target_name); } @@ -904,8 +914,8 @@ impl TypeChecker { }); } } - if manifest.exports.type_aliases.iter().any(|item| item.name == name) { - return Some(ManifestExportRef::TypeAlias); + if let Some(item) = manifest.exports.type_aliases.iter().find(|item| item.name == name) { + return Some(ManifestExportRef::TypeAlias(item)); } if let Some(item) = manifest.exports.newtypes.iter().find(|item| item.name == name) { return Some(ManifestExportRef::Newtype(item)); @@ -919,6 +929,7 @@ impl TypeChecker { None } + /// Return whether a manifest export introduces a type-like name into the importing module. fn manifest_export_is_type(export: &ManifestExportRef<'_>) -> bool { matches!( export, @@ -926,7 +937,7 @@ impl TypeChecker { | ManifestExportRef::Class(_) | ManifestExportRef::Trait(_) | ManifestExportRef::Enum(_) - | ManifestExportRef::TypeAlias + | ManifestExportRef::TypeAlias(_) | ManifestExportRef::Newtype(_) ) } @@ -974,7 +985,18 @@ impl TypeChecker { enum_name: enum_name.to_string(), fields: fields.iter().map(resolved_type_from_manifest_type_ref).collect(), }), - ManifestExportRef::TypeAlias => SymbolKind::Type(TypeInfo::TypeAlias), + ManifestExportRef::TypeAlias(export) => { + let mut target = resolved_type_from_manifest_type_ref(&export.target); + Self::remap_resolved_type_with_import_aliases(&mut target, imported_type_aliases); + self.type_aliases.insert( + local_name.clone(), + crate::frontend::typechecker::TypeAliasTarget { + type_params: export.type_params.iter().map(|param| param.name.clone()).collect(), + target, + }, + ); + SymbolKind::Type(TypeInfo::TypeAlias) + } ManifestExportRef::Newtype(export) => { SymbolKind::Type(TypeInfo::Newtype(self.newtype_info_from_manifest(export))) } @@ -990,13 +1012,23 @@ impl TypeChecker { is_used: false, }), ManifestExportRef::Alias(export) => { - let Some(target_name) = export.target_path.last() else { - return; - }; - let Some(target_export) = Self::find_manifest_export(manifest, target_name) else { - return; - }; - return self.define_pub_import_symbol(manifest, local_name, target_export, imported_type_aliases, span); + if let Some(function) = &export.projected_function { + SymbolKind::Function(self.function_info_from_manifest(function)) + } else { + let Some(target_name) = export.target_path.last() else { + return; + }; + let Some(target_export) = Self::find_manifest_export(manifest, target_name) else { + return; + }; + return self.define_pub_import_symbol( + manifest, + local_name, + target_export, + imported_type_aliases, + span, + ); + } } }; self.remap_symbol_kind_with_import_aliases(&mut kind, imported_type_aliases); @@ -1579,7 +1611,7 @@ impl TypeChecker { } if trait_methods.is_empty() { trait_methods.extend( - Self::known_rust_trait_methods(info.path.as_str()) + fallback_rust_trait_methods(info.path.as_str()) .iter() .map(|method| (*method).to_string()), ); @@ -1601,71 +1633,6 @@ impl TypeChecker { self.define_rust_import_symbol(name, info, span); } - /// Return fallback trait method names for Rust traits when rustdoc metadata is unavailable. - fn known_rust_trait_methods(path: &str) -> &'static [&'static str] { - match path { - "std::io::Read" => &[ - "read", - "read_to_end", - "read_to_string", - "read_exact", - "read_buf", - "read_buf_exact", - "bytes", - "chain", - "take", - ], - "std::io::Write" => &["write", "write_all", "write_fmt", "flush"], - "std::io::Seek" => &["seek", "rewind", "stream_position", "seek_relative"], - "byteorder::ReadBytesExt" => &[ - "read_u8", - "read_i8", - "read_u16", - "read_i16", - "read_u32", - "read_i32", - "read_u64", - "read_i64", - "read_u128", - "read_i128", - "read_f32", - "read_f64", - ], - "byteorder::WriteBytesExt" => &[ - "write_u8", - "write_i8", - "write_u16", - "write_i16", - "write_u32", - "write_i32", - "write_u64", - "write_i64", - "write_u128", - "write_i128", - "write_f32", - "write_f64", - ], - "sha2::Digest" | "sha3::Digest" | "blake2::Digest" | "md5::Digest" | "sha1::Digest" => &[ - "new", - "new_with_prefix", - "update", - "chain_update", - "finalize", - "finalize_into", - "finalize_reset", - "reset", - "output_size", - "digest", - ], - "blake2::digest::XofReader" | "sha3::digest::XofReader" => &["read"], - "std::os::unix::fs::MetadataExt" => &[ - "dev", "ino", "mode", "nlink", "uid", "gid", "rdev", "size", "atime", "mtime", "ctime", "blksize", - "blocks", - ], - _ => &[], - } - } - /// Define a symbol for a Rust crate import. /// /// Explicit Rust imports must be allowed to shadow dependency-exported Incan types with the same simple name. This @@ -1698,8 +1665,7 @@ impl TypeChecker { fn existing_from_import_symbol_kind(&self, name: &str) -> Option { let id = self.symbols.lookup(name)?; let sym = self.symbols.get(id)?; - let is_implicit_builtin = sym.scope == 0 && sym.span == Span::default(); - if is_implicit_builtin { + if Self::is_implicit_builtin_symbol(sym) { return None; } Some(sym.kind.clone()) diff --git a/src/frontend/typechecker/const_eval.rs b/src/frontend/typechecker/const_eval.rs index 80c4841a2..799727cc1 100644 --- a/src/frontend/typechecker/const_eval.rs +++ b/src/frontend/typechecker/const_eval.rs @@ -793,6 +793,12 @@ impl TypeChecker { } Expr::Call(callee, type_args, args) if type_args.is_empty() => { + if let Expr::Ident(callee_name) = &callee.node + && self.is_const_model_constructor_name(callee_name) + { + return self.eval_const_model_constructor(callee_name, args, expected, stack, decl_span, expr.span); + } + let Some(ResolvedType::Named(expected_name)) = expected else { self.errors.push(errors::const_expression_not_allowed(expr.span)); return None; @@ -834,6 +840,9 @@ impl TypeChecker { value: None, }) } + Expr::Constructor(name, args) if self.is_const_model_constructor_name(name) => { + self.eval_const_model_constructor(name, args, expected, stack, decl_span, expr.span) + } // Disallowed constructs for RFC 008 phase 1. Expr::Call(_, _, _) @@ -864,6 +873,144 @@ impl TypeChecker { } } + /// Return whether a name resolves to a model constructor that can be considered for const literal validation. + fn is_const_model_constructor_name(&self, name: &str) -> bool { + self.lookup_type_info(name) + .is_some_and(|info| matches!(info, TypeInfo::Model(_))) + } + + /// Evaluate a model constructor in a const initializer. + /// + /// This keeps `const` model literals declaration-safe: every provided field must itself be const-evaluable, all + /// required fields must be explicit, and omitted defaults are rejected because model defaults are ordinary runtime + /// expressions rather than const metadata. + fn eval_const_model_constructor( + &mut self, + type_name: &str, + args: &[CallArg], + expected: Option<&ResolvedType>, + stack: &mut Vec, + decl_span: Span, + call_span: Span, + ) -> Option { + if let Some(expected_ty) = expected + && !matches!(expected_ty, ResolvedType::Named(name) if name == type_name) + && !matches!(expected_ty, ResolvedType::Unknown) + { + return Some(ConstEvalResult { + ty: ResolvedType::Named(type_name.to_string()), + kind: ConstKind::RustNative, + value: None, + }); + } + + let Some(TypeInfo::Model(model)) = self.lookup_type_info(type_name).cloned() else { + self.errors.push(errors::const_expression_not_allowed(call_span)); + return None; + }; + + let mut provided = std::collections::HashSet::new(); + let mut result_kind = ConstKind::RustNative; + let mut had_error = false; + + for arg in args { + let CallArg::Named(field_name, value) = arg else { + self.errors + .push(errors::positional_constructor_args_not_supported(type_name, call_span)); + had_error = true; + continue; + }; + + let Some((canonical_name, field_info)) = Self::resolve_const_model_field(&model.fields, field_name) else { + self.eval_const_expr(value, None, stack, decl_span); + self.errors + .push(errors::missing_field(type_name, field_name, value.span)); + had_error = true; + continue; + }; + + if !provided.insert(canonical_name.clone()) { + self.errors.push(errors::duplicate_field_in_call( + type_name, + canonical_name.as_str(), + value.span, + )); + had_error = true; + continue; + } + + let Some(field_result) = self.eval_const_expr(value, Some(&field_info.ty), stack, decl_span) else { + had_error = true; + continue; + }; + if field_result.kind == ConstKind::Frozen { + result_kind = ConstKind::Frozen; + } + + if field_result.ty != field_info.ty { + match self.const_int_value_checked_against_numeric_expected(&field_result, &field_info.ty, value.span) { + Some(true) => {} + Some(false) => had_error = true, + None => { + self.errors.push(errors::field_type_mismatch( + field_name, + &field_info.ty.to_string(), + &field_result.ty.to_string(), + value.span, + )); + had_error = true; + } + } + } + } + + for (field_name, field_info) in &model.fields { + if provided.contains(field_name) { + continue; + } + if field_info.has_default { + self.errors.push(CompileError::type_error( + format!( + "Const model constructor '{}' must provide field '{}' explicitly; model defaults are not evaluated in const initializers", + type_name, field_name + ), + call_span, + )); + } else { + self.errors.push(errors::missing_required_constructor_field( + type_name, field_name, call_span, + )); + } + had_error = true; + } + + if had_error { + return None; + } + + Some(ConstEvalResult { + ty: ResolvedType::Named(type_name.to_string()), + kind: result_kind, + value: None, + }) + } + + /// Resolve a model constructor field by canonical source name or model alias. + fn resolve_const_model_field<'a>( + fields: &'a std::collections::HashMap, + field_name: &str, + ) -> Option<(String, &'a crate::frontend::symbols::FieldInfo)> { + fields + .get(field_name) + .map(|info| (field_name.to_string(), info)) + .or_else(|| { + fields + .iter() + .find(|(_, info)| info.alias.as_deref() == Some(field_name)) + .map(|(name, info)| (name.clone(), info)) + }) + } + /// Evaluate a literal in a const context, optionally checking it against an expected type. fn eval_const_literal( &mut self, diff --git a/src/frontend/typechecker/helpers/symbols.rs b/src/frontend/typechecker/helpers/symbols.rs index 56093f40c..4432a44f8 100644 --- a/src/frontend/typechecker/helpers/symbols.rs +++ b/src/frontend/typechecker/helpers/symbols.rs @@ -4,17 +4,37 @@ //! expression checking make the same shadowing decision. use crate::frontend::ast::Span; -use crate::frontend::symbols::SymbolKind; +use crate::frontend::symbols::Symbol; use crate::frontend::typechecker::TypeChecker; +use incan_core::lang::surface::functions::SurfaceFnId; +use incan_core::lang::surface::types::SurfaceTypeId; impl TypeChecker { - /// Return `true` when `name` resolves to a non-builtin function definition. + /// Return whether a symbol is one of the ambient builtins seeded into the root symbol table before source + /// collection. + pub(in crate::frontend::typechecker) fn is_implicit_builtin_symbol(sym: &Symbol) -> bool { + sym.scope == 0 && sym.span == Span::default() + } + + /// Return `true` when an implicit builtin-call root is shadowed by a real source/import binding. /// - /// Call checking uses this to decide whether builtin dispatch should yield to a user/imported function of the same - /// name. - pub(in crate::frontend::typechecker) fn has_non_builtin_function_definition(&self, name: &str) -> bool { - self.lookup_symbol(name).is_some_and(|sym| { - matches!(sym.kind, SymbolKind::Function(_)) && !(sym.scope == 0 && sym.span == Span::default()) - }) + /// Decorated functions are intentionally rebound from `Function` symbols to callable `Variable` symbols after + /// decorator checking. Builtin dispatch therefore has to ask whether the name is still the ambient builtin binding, + /// not whether the symbol is specifically a `Function`. + pub(in crate::frontend::typechecker) fn has_non_builtin_call_root_binding(&self, name: &str) -> bool { + self.lookup_symbol(name) + .is_some_and(|sym| !Self::is_implicit_builtin_symbol(sym)) + } + + /// Return the active stdlib surface helper imported under `name`, if the import has not been shadowed. + pub(in crate::frontend::typechecker) fn active_surface_function_import(&self, name: &str) -> Option { + let (id, imported_symbol_id) = self.surface_function_import_bindings.get(name)?; + (self.symbols.lookup(name) == Some(*imported_symbol_id)).then_some(*id) + } + + /// Return the active stdlib surface type imported under `name`, if the import has not been shadowed. + pub(in crate::frontend::typechecker) fn active_surface_type_import(&self, name: &str) -> Option { + let (id, imported_symbol_id) = self.surface_type_import_bindings.get(name)?; + (self.symbols.lookup(name) == Some(*imported_symbol_id)).then_some(*id) } } diff --git a/src/frontend/typechecker/mod.rs b/src/frontend/typechecker/mod.rs index 2fb2a7986..c0b1ab53a 100644 --- a/src/frontend/typechecker/mod.rs +++ b/src/frontend/typechecker/mod.rs @@ -49,15 +49,17 @@ mod collect; mod const_eval; mod helpers; pub(crate) mod stdlib_loader; +mod trait_bound_relations; mod type_info; mod validate_rust_module; pub use const_eval::ConstValue; pub use type_info::{ - ComputedPropertyAccessInfo, DecoratedFunctionBindingInfo, DecoratedMethodBindingInfo, FixedUnpackPlan, IdentKind, - ProtocolIterationInfo, ResolvedMethodCall, ResolvedMethodDispatch, ResolvedOperatorCall, ResolvedOperatorKind, - RustArgCoercionInfo, RustArgCoercionKind, StaticBindingInfo, TestingFixtureInfo, TypeCheckInfo, - ValidatedNewtypeCoercionInfo, ValidatedNewtypeCoercionMode, ValidatedNewtypeCoercionStep, + ComputedPropertyAccessInfo, DecoratedFunctionBindingInfo, DecoratedMethodBindingInfo, FixedUnpackPlan, + FunctionBindingInfo, IdentKind, ProtocolIterationInfo, ResolvedMethodCall, ResolvedMethodDispatch, + ResolvedOperatorCall, ResolvedOperatorKind, RustArgCoercionInfo, RustArgCoercionKind, StaticBindingInfo, + TestingFixtureInfo, TypeCheckInfo, ValidatedNewtypeCoercionInfo, ValidatedNewtypeCoercionMode, + ValidatedNewtypeCoercionStep, }; #[cfg(test)] mod tests; @@ -76,12 +78,16 @@ use crate::frontend::surface_semantics::SurfaceContext; use crate::frontend::symbols::*; #[cfg(feature = "rust_inspect")] use crate::rust_inspect::RustMetadataCache; -use helpers::{collection_type_id, render_resolved_type_as_rust_arg, stringlike_type_id}; -use incan_core::interop::{RustFunctionSig, RustItemKind, RustItemMetadata, RustParam, RustTypeShape}; +use helpers::{collection_name, collection_type_id, render_resolved_type_as_rust_arg, stringlike_type_id}; +use incan_core::interop::{ + RustFunctionSig, RustItemKind, RustItemMetadata, RustParam, RustTypeShape, render_rust_type_shape_path, + split_top_level_rust_args, strip_rust_borrow_lifetimes, +}; use incan_core::lang::conventions; use incan_core::lang::decorators::{self as core_decorators, DecoratorId}; +use incan_core::lang::surface::functions::SurfaceFnId; use incan_core::lang::surface::types as surface_types; -use incan_core::lang::surface::types::SurfaceTypeKind; +use incan_core::lang::surface::types::{SurfaceTypeId, SurfaceTypeKind}; use incan_core::lang::traits::{self as builtin_traits, TraitId}; use incan_core::lang::types::collections::CollectionTypeId; use incan_core::lang::types::numerics::{self, NumericFamily, NumericTypeId}; @@ -228,6 +234,16 @@ pub struct TypeChecker { /// These names are disallowed in runtime call expressions; markers are decorator-only semantics consumed by the /// test runner. pub(crate) testing_marker_import_bindings: HashSet, + /// Local names bound to stdlib surface helpers that still need compiler-known call typing. + /// + /// The stored symbol id must remain the active lookup binding; later local declarations or user imports with the + /// same name shadow these helper semantics. + pub(crate) surface_function_import_bindings: HashMap, + /// Local names bound to stdlib surface types that still need compiler-known constructor typing. + /// + /// The stored symbol id must remain the active lookup binding; later local declarations or user imports with the + /// same name shadow these constructor semantics. + pub(crate) surface_type_import_bindings: HashMap, /// Fixture function names collected before body checking so dependency metadata is order-independent. pub(crate) testing_fixture_names: HashSet, /// Import aliases collected from `import` / `from ... import` declarations. @@ -299,6 +315,8 @@ impl TypeChecker { declared_crate_names: None, stdlib_cache: stdlib_loader::StdlibAstCache::new(), testing_marker_import_bindings: HashSet::new(), + surface_function_import_bindings: HashMap::new(), + surface_type_import_bindings: HashMap::new(), testing_fixture_names: HashSet::new(), import_aliases: HashMap::new(), surface_context: SurfaceContext::default(), @@ -477,25 +495,7 @@ impl TypeChecker { } fn split_top_level_generic_args(args: &str) -> Vec<&str> { - let mut parts = Vec::new(); - let mut depth = 0usize; - let mut start = 0usize; - for (idx, ch) in args.char_indices() { - match ch { - '<' | '(' | '[' => depth += 1, - '>' | ')' | ']' => depth = depth.saturating_sub(1), - ',' if depth == 0 => { - parts.push(args[start..idx].trim()); - start = idx + ch.len_utf8(); - } - _ => {} - } - } - let tail = args[start..].trim(); - if !tail.is_empty() { - parts.push(tail); - } - parts + split_top_level_rust_args(args) } /// Normalize a rust-inspect lookup path down to the nominal item path. @@ -634,15 +634,32 @@ impl TypeChecker { } return Some(false); } + Some(self.rust_type_args_compatible(actual_args.as_slice(), expected_args.as_slice())) + } + + fn rust_type_args_compatible(&self, actual_args: &[ResolvedType], expected_args: &[ResolvedType]) -> bool { if actual_args.len() != expected_args.len() { - return Some(actual_args.is_empty() && expected_args.is_empty()); + return (actual_args.is_empty() && expected_args.iter().all(Self::rust_type_arg_is_unknown_placeholder)) + || (expected_args.is_empty() && actual_args.iter().all(Self::rust_type_arg_is_unknown_placeholder)); + } + actual_args.iter().zip(expected_args.iter()).all(|(actual, expected)| { + Self::rust_type_arg_is_unknown_placeholder(actual) + || Self::rust_type_arg_is_unknown_placeholder(expected) + || self.types_compatible(actual, expected) + }) + } + + fn rust_type_arg_is_unknown_placeholder(arg: &ResolvedType) -> bool { + match arg { + ResolvedType::Unknown => true, + ResolvedType::RustPath(path) => { + matches!( + path.trim().as_bytes(), + [b'?'] | [b'{', b'u', b'n', b'k', b'n', b'o', b'w', b'n', b'}'] + ) + } + _ => false, } - Some( - actual_args - .iter() - .zip(expected_args.iter()) - .all(|(actual, expected)| self.types_compatible(actual, expected)), - ) } /// Whether a Rust signature parameter is the implicit receiver (`self`/`&self`/`&mut self`). @@ -740,53 +757,107 @@ impl TypeChecker { /// /// When `args` is empty, returns `path` unchanged (no angle brackets). fn render_rust_shape_path(path: &str, args: &[RustTypeShape]) -> String { - if args.is_empty() { - return path.to_string(); - } - let rendered_args: Vec = args.iter().map(Self::render_rust_shape_type).collect(); - format!("{path}<{}>", rendered_args.join(", ")) + render_rust_type_shape_path(path, args) } - /// Pretty-print a [`RustTypeShape`] as a stable Rust-like type string. + /// Detect whether a Rust display type starts with `&T` or `&mut T`. /// - /// Feeds [`ResolvedType::RustPath`] strings. Scalar widths are normalized (`f64`, `i64`, `String`, `Vec`) to - /// match [`Self::resolved_type_from_rust_shape`], not to recover the exact original Rust spelling from metadata. - fn render_rust_shape_type(shape: &RustTypeShape) -> String { - match shape { - RustTypeShape::Bool => "bool".to_string(), - RustTypeShape::Float => "f64".to_string(), - RustTypeShape::Int => "i64".to_string(), - RustTypeShape::Str => "String".to_string(), - RustTypeShape::Bytes => "Vec".to_string(), - RustTypeShape::Unit => "()".to_string(), - RustTypeShape::Option(inner) => format!("Option<{}>", Self::render_rust_shape_type(inner)), - RustTypeShape::Result(ok, err) => { - format!( - "Result<{}, {}>", - Self::render_rust_shape_type(ok), - Self::render_rust_shape_type(err) - ) - } - RustTypeShape::Tuple(items) => { - let rendered: Vec = items.iter().map(Self::render_rust_shape_type).collect(); - format!("({})", rendered.join(", ")) - } - RustTypeShape::Ref(inner) => format!("&{}", Self::render_rust_shape_type(inner)), - RustTypeShape::RustPath { path, args } => Self::render_rust_shape_path(path, args), - RustTypeShape::TypeParam(name) => name.clone(), - RustTypeShape::Unknown => "?".to_string(), + /// Returns the mutability flag plus the remaining inner type spelling so [`Self::resolved_type_from_rust_display`] + /// can preserve borrow semantics for Rust-backed values instead of collapsing them into plain path types. + fn rust_display_borrow_kind(display: &str) -> Option<(bool, &str)> { + let after_amp = display.trim().strip_prefix('&')?.trim_start(); + if let Some(inner) = after_amp.strip_prefix("mut") + && inner.chars().next().is_none_or(char::is_whitespace) + { + return Some((true, inner.trim_start())); } + Some((false, after_amp)) } - /// Detect whether a normalized Rust display type starts with `&T` or `&mut T`. + /// Remove Rust lifetime labels that decorate borrowed display types. /// - /// Returns the mutability flag plus the remaining inner type spelling so [`Self::resolved_type_from_rust_display`] - /// can preserve borrow semantics for Rust-backed values instead of collapsing them into plain path types. - fn rust_display_borrow_kind(normalized: &str) -> Option<(bool, &str)> { - if let Some(inner) = normalized.strip_prefix("&mut") { - return Some((true, inner)); + /// rust-analyzer commonly prints borrowed method parameters as `&'h str` or `&'a [u8]`. The typechecker only needs + /// the ownership shape and payload type, so erase the lifetime after `&` before whitespace normalization turns it + /// into an unparseable token such as `&'hstr`. + fn strip_borrow_lifetimes(rust_ty: &str) -> String { + strip_rust_borrow_lifetimes(rust_ty) + } + + fn rust_display_without_lifetimes(rust_ty: &str) -> String { + Self::strip_borrow_lifetimes(rust_ty) + .replace("'static ", "") + .replace("'_", "") + .trim_start_matches("::") + .to_string() + } + + fn compact_rust_display(rust_ty: &str) -> String { + Self::rust_display_without_lifetimes(rust_ty).replace(' ', "") + } + + fn rust_generic_base_and_args(normalized: &str) -> Option<(&str, Vec<&str>)> { + let start = normalized.find('<')?; + if !normalized.ends_with('>') { + return None; + } + let base = normalized[..start].trim(); + let inner = &normalized[start + 1..normalized.len() - 1]; + Some((base, Self::split_top_level_generic_args(inner))) + } + + fn rust_collection_id_from_display_base(base: &str) -> Option { + incan_core::lang::types::collections::from_rust_display_base(base) + } + + fn resolved_structural_rust_param_display(&self, normalized: &str, mut resolve_arg: F) -> Option + where + F: FnMut(&Self, &str) -> ResolvedType, + { + let (base, arg_displays) = Self::rust_generic_base_and_args(normalized)?; + let collection_id = Self::rust_collection_id_from_display_base(base)?; + let mut args = arg_displays + .into_iter() + .map(|arg| resolve_arg(self, arg)) + .collect::>(); + match collection_id { + CollectionTypeId::List if args.len() == 1 => Some(ResolvedType::Generic( + collection_name(CollectionTypeId::List).to_string(), + args, + )), + CollectionTypeId::Dict if args.len() == 2 => Some(ResolvedType::Generic( + collection_name(CollectionTypeId::Dict).to_string(), + args, + )), + CollectionTypeId::Set if args.len() == 1 => Some(ResolvedType::Generic( + collection_name(CollectionTypeId::Set).to_string(), + args, + )), + CollectionTypeId::Option if args.len() == 1 => Some(ResolvedType::Generic( + collection_name(CollectionTypeId::Option).to_string(), + args, + )), + CollectionTypeId::Result if args.len() <= 2 => { + let ok = args.first().cloned().unwrap_or(ResolvedType::Unknown); + let err = args.get(1).cloned().unwrap_or(ResolvedType::Unknown); + Some(ResolvedType::Generic( + collection_name(CollectionTypeId::Result).to_string(), + vec![ok, err], + )) + } + CollectionTypeId::Tuple => Some(ResolvedType::Tuple(args)), + CollectionTypeId::FrozenList if args.len() == 1 => Some(ResolvedType::FrozenList(Box::new(args.remove(0)))), + CollectionTypeId::FrozenSet if args.len() == 1 => Some(ResolvedType::FrozenSet(Box::new(args.remove(0)))), + CollectionTypeId::FrozenDict if args.len() == 2 => { + let value = args.pop().unwrap_or(ResolvedType::Unknown); + let key = args.pop().unwrap_or(ResolvedType::Unknown); + Some(ResolvedType::FrozenDict(Box::new(key), Box::new(value))) + } + CollectionTypeId::Generator if args.len() == 1 => Some(ResolvedType::Generic( + collection_name(CollectionTypeId::Generator).to_string(), + args, + )), + _ => None, } - normalized.strip_prefix('&').map(|inner| (false, inner)) } /// Map structured rust-inspect [`RustTypeShape`] into a [`ResolvedType`] for field access and pattern typing. @@ -855,13 +926,12 @@ impl TypeChecker { /// /// ## `Result` parsing /// - /// `Result<…>` is split on the **first** top-level comma only. Nested generics that contain commas (for example - /// `Result, String>`) are therefore parsed incorrectly and may degrade to [`ResolvedType::Unknown`] - /// for one or both type arguments. Prefer precise typing from Incan surfaces over relying on this heuristic. + /// `Result<…>` uses top-level generic splitting, so nested generic or tuple commas stay inside the appropriate + /// argument. Prefer precise typing from Incan surfaces over relying on this heuristic for arbitrary Rust paths. pub(crate) fn resolved_type_from_rust_display(&self, rust_ty: &str) -> ResolvedType { let trimmed = rust_ty.trim(); - let no_lifetimes = trimmed.replace("'static ", "").replace("'_", "").replace(' ', ""); - let normalized = no_lifetimes.trim_start_matches("::").to_string(); + let display = Self::rust_display_without_lifetimes(trimmed); + let normalized = display.replace(' ', ""); if let Some(Symbol { kind: SymbolKind::RustItem(info), .. @@ -875,7 +945,7 @@ impl TypeChecker { "&[u8]" => return ResolvedType::Bytes, _ => {} } - if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(normalized.as_str()) { + if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(display.as_str()) { let inner_ty = self.resolved_type_from_rust_display(inner); return if is_mut { ResolvedType::RefMut(Box::new(inner_ty)) @@ -884,6 +954,7 @@ impl TypeChecker { }; } match normalized.as_str() { + "{unknown}" => ResolvedType::Unknown, "bool" => ResolvedType::Bool, "f64" => ResolvedType::Float, "i64" => ResolvedType::Int, @@ -903,31 +974,32 @@ impl TypeChecker { "Vec" | "std::vec::Vec" | "alloc::vec::Vec" | "&[u8]" => ResolvedType::Bytes, "()" => ResolvedType::Unit, _ if normalized.ends_with('>') => { - if let Some((base, inner)) = normalized.split_once('<') { - let base = base.trim_end_matches('>'); - let inner = inner.trim_end_matches('>'); + if let Some((base, args)) = Self::rust_generic_base_and_args(normalized.as_str()) { let tail = base.rsplit("::").next().unwrap_or(base); match collection_type_id(tail) { Some(CollectionTypeId::Option) => { + let inner = args.first().copied().unwrap_or(""); return ResolvedType::Generic( - "Option".to_string(), + collection_name(CollectionTypeId::Option).to_string(), vec![self.resolved_type_from_rust_display(inner)], ); } Some(CollectionTypeId::Result) => { - let mut parts = inner.splitn(2, ','); - let ok_ty = parts - .next() + let ok_ty = args + .first() .map(|p| self.resolved_type_from_rust_display(p)) .unwrap_or(ResolvedType::Unknown); // Result aliases such as `datafusion_common::error::Result` often erase the concrete // error arm from the display. Keep the success path semantic and degrade only the missing // error arm. - let err_ty = parts - .next() + let err_ty = args + .get(1) .map(|p| self.resolved_type_from_rust_display(p)) .unwrap_or(ResolvedType::Unknown); - return ResolvedType::Generic("Result".to_string(), vec![ok_ty, err_ty]); + return ResolvedType::Generic( + collection_name(CollectionTypeId::Result).to_string(), + vec![ok_ty, err_ty], + ); } _ => {} } @@ -949,9 +1021,18 @@ impl TypeChecker { } } - /// Return a Rust generic type-parameter name when the display is the simple identifier form rust-analyzer uses - /// for params like `T` or `U`. + /// Return a Rust generic parameter display when rust-analyzer reports a by-value generic boundary. + /// + /// Plain type parameters appear as `T` or `U`. Anonymous `impl Trait` parameters can arrive with whitespace erased, + /// such as `implBuf` for `impl Buf`; those still carry by-value shape and must not be treated as borrowed Rust + /// boundary targets. pub(crate) fn rust_display_type_var_name(normalized: &str) -> Option<&str> { + if let Some(tail) = normalized.strip_prefix("impl") + && !tail.is_empty() + && (tail.contains("::") || tail.chars().next().is_some_and(|ch| ch.is_ascii_uppercase())) + { + return Some(normalized); + } if normalized.len() == 1 && normalized.chars().next().is_some_and(|ch| ch.is_ascii_uppercase()) { Some(normalized) } else { @@ -966,16 +1047,17 @@ impl TypeChecker { /// borrowed Rust boundary so emission can pass `&arg` instead of moving an owned `String` or `Vec`. pub(crate) fn resolved_param_type_from_rust_display(&self, rust_ty: &str) -> ResolvedType { let trimmed = rust_ty.trim(); - let no_lifetimes = trimmed.replace("'static ", "").replace("'_", "").replace(' ', ""); - let normalized = no_lifetimes.trim_start_matches("::").to_string(); + let display = Self::rust_display_without_lifetimes(trimmed); + let normalized = display.replace(' ', ""); if let Some(name) = Self::rust_display_type_var_name(normalized.as_str()) { return ResolvedType::TypeVar(name.to_string()); } - if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(normalized.as_str()) { + if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(display.as_str()) { + let inner_normalized = Self::compact_rust_display(inner); let inner_ty = match inner { "str" => ResolvedType::Str, "[u8]" => ResolvedType::Bytes, - _ => self.resolved_type_from_rust_display(inner), + _ => self.resolved_type_from_rust_display(inner_normalized.as_str()), }; return if is_mut { ResolvedType::RefMut(Box::new(inner_ty)) @@ -983,9 +1065,47 @@ impl TypeChecker { ResolvedType::Ref(Box::new(inner_ty)) }; } + if let Some(structural) = self.resolved_structural_rust_param_display(normalized.as_str(), |checker, arg| { + checker.resolved_param_type_from_rust_display(arg) + }) { + return structural; + } self.resolved_type_from_rust_display(normalized.as_str()) } + /// Convert a Rust parameter display type into the typed target carried by Rust-boundary coercion metadata. + /// + /// This preserves the semantic difference between slice borrow targets such as `&str`/`&[u8]` and borrowed owned + /// Rust targets such as `&String`/`&Vec`. Ordinary parameter typing still maps Rust scalar displays onto Incan + /// value types, but coercion metadata is a backend contract: lowering and emission must be able to choose borrow + /// versus materialize-then-borrow behavior from this typed target without re-decoding Rust display strings. + pub(crate) fn resolved_rust_boundary_target_from_param_display(&self, rust_ty: &str) -> ResolvedType { + let trimmed = rust_ty.trim(); + let display = Self::rust_display_without_lifetimes(trimmed); + let normalized = display.replace(' ', ""); + if let Some((is_mut, inner)) = Self::rust_display_borrow_kind(display.as_str()) { + let inner_normalized = Self::compact_rust_display(inner); + let inner_ty = match inner { + "str" => ResolvedType::Str, + "[u8]" => ResolvedType::Bytes, + "String" | "std::string::String" | "alloc::string::String" => ResolvedType::RustPath(inner.to_string()), + "Vec" | "std::vec::Vec" | "alloc::vec::Vec" => ResolvedType::RustPath(inner.to_string()), + _ => self.resolved_type_from_rust_display(inner_normalized.as_str()), + }; + return if is_mut { + ResolvedType::RefMut(Box::new(inner_ty)) + } else { + ResolvedType::Ref(Box::new(inner_ty)) + }; + } + if let Some(structural) = self.resolved_structural_rust_param_display(normalized.as_str(), |checker, arg| { + checker.resolved_rust_boundary_target_from_param_display(arg) + }) { + return structural; + } + self.resolved_param_type_from_rust_display(normalized.as_str()) + } + /// Set the declared Rust crate names from `incan.toml [rust-dependencies]`. /// /// When set, `rust.module()` path validation will check that the first segment of the path is either `incan_stdlib` @@ -1698,7 +1818,7 @@ impl TypeChecker { } Expr::FString(parts) => { for part in parts { - if let FStringPart::Expr(expr) = part { + if let FStringPart::Expr { expr, .. } = part { self.collect_static_dependencies_from_expr(&expr.node, deps, visiting_functions); } } @@ -1876,7 +1996,7 @@ impl TypeChecker { } Expr::FString(parts) => { for part in parts { - if let FStringPart::Expr(inner) = part { + if let FStringPart::Expr { expr: inner, .. } = part { self.collect_static_initializer_static_writes_from_expr( inner, current_static, @@ -3203,6 +3323,8 @@ impl TypeChecker { self.warnings.clear(); self.errors.clear(); self.testing_marker_import_bindings.clear(); + self.surface_function_import_bindings.clear(); + self.surface_type_import_bindings.clear(); self.testing_fixture_names.clear(); self.surface_context = SurfaceContext::from_program(program); self.import_aliases = self.surface_context.import_aliases().clone(); @@ -3773,6 +3895,12 @@ impl TypeChecker { } }; + let expanded_actual = self.expand_type_aliases(actual.clone()); + let expanded_expected = self.expand_type_aliases(expected.clone()); + if &expanded_actual != actual || &expanded_expected != expected { + return self.types_compatible(&expanded_actual, &expanded_expected); + } + if actual == expected { return true; } diff --git a/src/frontend/typechecker/tests.rs b/src/frontend/typechecker/tests.rs index 7425f8c94..4535c495f 100644 --- a/src/frontend/typechecker/tests.rs +++ b/src/frontend/typechecker/tests.rs @@ -12,10 +12,11 @@ use crate::frontend::library_manifest_index::{ use crate::frontend::testing_markers::TestingFixtureScope; use crate::frontend::{lexer, parser}; use crate::library_manifest::{ - ClassExport, ConstExport, EnumExport, EnumValueExport, EnumValueTypeExport, EnumVariantExport, FunctionExport, - LibraryContractMetadata, LibraryExports, LibraryManifest, LibraryRustAbi, MethodExport, ModelExport, ParamExport, - ParamKindExport, PartialExport, PartialPresetExport, PartialTargetKindExport, PresetValueExport, ReceiverExport, - StaticExport, TraitExport, TypeBoundExport, TypeParamExport, TypeRef, + AliasExport, ClassExport, ConstExport, EnumExport, EnumValueExport, EnumValueTypeExport, EnumVariantExport, + FunctionExport, LibraryContractMetadata, LibraryExports, LibraryManifest, LibraryRustAbi, MethodExport, + ModelExport, ParamExport, ParamKindExport, PartialExport, PartialPresetExport, PartialTargetKindExport, + PresetValueExport, ReceiverExport, StaticExport, TraitExport, TypeAliasExport, TypeBoundExport, TypeParamExport, + TypeRef, }; #[cfg(feature = "rust_inspect")] use crate::rust_inspect::{Inspector, InspectorConfig, write_borrowed_param_probe_crate, write_substrait_probe_crate}; @@ -528,6 +529,37 @@ def use() -> str: .unwrap_or_else(|errs| panic!("consumer should import public partial callable: {errs:?}")); } +#[test] +fn test_from_import_accepts_public_partial_export() { + let library = parse_program( + r#" +pub model Spec: + namespace: str + policy: str + klass: str + lifecycle: str + +pub core_spec = partial Spec(namespace="core", policy="portable") +"#, + "partial import library", + ); + let consumer = parse_program( + r#" +from presets import core_spec + +def use() -> str: + spec = core_spec(klass="scalar", lifecycle="v1") + return spec.namespace +"#, + "partial from-import consumer", + ); + + let mut checker = TypeChecker::new(); + checker + .check_with_imports(&consumer, &[("presets", &library)]) + .unwrap_or_else(|errs| panic!("consumer should import public partial callable by name: {errs:?}")); +} + #[test] fn test_method_partial_presets_project_as_defaults_for_trait_and_model() { let source = r#" @@ -1206,6 +1238,10 @@ pub mean = alias avg assert_eq!(manifest.exports.aliases.len(), 1); assert_eq!(manifest.exports.aliases[0].name, "mean"); assert_eq!(manifest.exports.aliases[0].target_path, vec!["avg"]); + assert!( + manifest.exports.aliases[0].projected_function.is_some(), + "function aliases should carry callable projection metadata for pub:: consumers" + ); assert!( manifest .exports @@ -1460,7 +1496,13 @@ fn library_index_with_mylib_exports() -> LibraryManifestIndex { }], derives: Vec::new(), }], - type_aliases: Vec::new(), + type_aliases: vec![TypeAliasExport { + name: "WidgetAlias".to_string(), + type_params: Vec::new(), + target: TypeRef::Named { + name: "Widget".to_string(), + }, + }], newtypes: Vec::new(), consts: vec![ConstExport { name: "DEFAULT_NAME".to_string(), @@ -1493,6 +1535,59 @@ fn library_index_with_mylib_exports() -> LibraryManifestIndex { )])) } +fn library_index_with_callable_alias_export() -> LibraryManifestIndex { + let manifest = LibraryManifest { + name: "mylib".to_string(), + version: "0.1.0".to_string(), + incan_version: crate::version::INCAN_VERSION.to_string(), + manifest_format: crate::library_manifest::LIBRARY_MANIFEST_FORMAT, + exports: LibraryExports { + aliases: vec![AliasExport { + name: "public_target".to_string(), + target_path: vec!["target_impl".to_string()], + projected_function: Some(FunctionExport { + name: "public_target".to_string(), + type_params: Vec::new(), + params: vec![ParamExport { + name: "value".to_string(), + ty: TypeRef::Named { + name: "int".to_string(), + }, + kind: ParamKindExport::Normal, + has_default: false, + }], + return_type: TypeRef::Named { + name: "int".to_string(), + }, + is_async: false, + }), + }], + partials: Vec::new(), + models: Vec::new(), + classes: Vec::new(), + functions: Vec::new(), + traits: Vec::new(), + enums: Vec::new(), + type_aliases: Vec::new(), + newtypes: Vec::new(), + consts: Vec::new(), + statics: Vec::new(), + }, + vocab: None, + soft_keywords: Default::default(), + contract_metadata: LibraryContractMetadata::default(), + rust_abi: None, + }; + + LibraryManifestIndex::from_entries(HashMap::from([( + "mylib".to_string(), + LibraryManifestIndexEntry::Loaded { + manifest: Box::new(manifest), + metadata: LibraryArtifactMetadata::from_crate_root("mylib", "mylib", synthetic_artifact_root("mylib")), + }, + )])) +} + fn library_index_with_trait_export() -> LibraryManifestIndex { let manifest = LibraryManifest { name: "mylib".to_string(), @@ -2511,6 +2606,8 @@ fn test_resolved_type_from_builtin_borrowed_displays_stays_stable() { let checker = TypeChecker::new(); assert_eq!(checker.resolved_type_from_rust_display("&str"), ResolvedType::Str); assert_eq!(checker.resolved_type_from_rust_display("&[u8]"), ResolvedType::Bytes); + assert_eq!(checker.resolved_type_from_rust_display("&'h str"), ResolvedType::Str); + assert_eq!(checker.resolved_type_from_rust_display("&'h [u8]"), ResolvedType::Bytes); } #[test] @@ -2524,6 +2621,61 @@ fn test_resolved_param_type_from_builtin_borrowed_displays_preserves_ref_payload checker.resolved_param_type_from_rust_display("&[u8]"), ResolvedType::Ref(Box::new(ResolvedType::Bytes)), ); + assert_eq!( + checker.resolved_param_type_from_rust_display("&'h str"), + ResolvedType::Ref(Box::new(ResolvedType::Str)), + ); + assert_eq!( + checker.resolved_param_type_from_rust_display("&'h [u8]"), + ResolvedType::Ref(Box::new(ResolvedType::Bytes)), + ); + assert_eq!( + checker.resolved_param_type_from_rust_display("&'h mut demo::Thing"), + ResolvedType::RefMut(Box::new(ResolvedType::RustPath("demo::Thing".to_string()))), + ); +} + +#[test] +fn test_resolved_param_type_from_structural_borrowed_display_preserves_nested_ref_payload() { + let checker = TypeChecker::new(); + assert_eq!( + checker.resolved_param_type_from_rust_display("Vec<&str>"), + ResolvedType::Generic("List".to_string(), vec![ResolvedType::Ref(Box::new(ResolvedType::Str))]), + ); + assert_eq!( + checker.resolved_rust_boundary_target_from_param_display("Vec<&String>"), + ResolvedType::Generic( + "List".to_string(), + vec![ResolvedType::Ref(Box::new(ResolvedType::RustPath( + "String".to_string() + )))] + ), + ); +} + +#[test] +fn test_resolved_param_type_does_not_treat_mut_prefix_as_mutable_borrow_keyword() { + let checker = TypeChecker::new(); + assert_eq!( + checker.resolved_param_type_from_rust_display("&mutability::Foo"), + ResolvedType::Ref(Box::new(ResolvedType::RustPath("mutability::Foo".to_string()))), + ); + assert_eq!( + checker.resolved_param_type_from_rust_display("&mut mutability::Foo"), + ResolvedType::RefMut(Box::new(ResolvedType::RustPath("mutability::Foo".to_string()))), + ); +} + +#[test] +fn test_resolved_result_display_splits_only_top_level_generic_commas() { + let checker = TypeChecker::new(); + assert_eq!( + checker.resolved_type_from_rust_display("Result, String>"), + ResolvedType::Generic( + "Result".to_string(), + vec![ResolvedType::RustPath("Vec<(i32,i32)>".to_string()), ResolvedType::Str,], + ), + ); } #[test] @@ -2816,6 +2968,49 @@ def normalize(value: int | str) -> str: ); } +#[test] +fn test_union_clone_method_typechecks_when_members_are_cloneable() { + let source = r#" +@derive(Clone) +model Leaf: + value: int + +@derive(Clone) +model Pair: + args: List[Expr] + +type Expr = Union[Leaf, Pair] + +def clone_expr(expr: Expr) -> Expr: + return expr.clone() +"#; + assert!(check_str(source).is_ok()); +} + +#[test] +fn test_union_model_variants_reject_direct_recursive_payload_without_indirection() { + let source = r#" +@derive(Clone) +model Leaf: + value: int + +@derive(Clone) +model Pair: + left: Expr + right: Expr + +type Expr = Union[Leaf, Pair] +"#; + let errors = check_str_err(source, "direct recursive union model payload should be rejected"); + assert!( + errors + .iter() + .any(|error| error.message.contains("direct recursive") && error.message.contains("Pair")), + "expected direct recursive model diagnostic, got: {:?}", + errors.iter().map(|error| &error.message).collect::>() + ); +} + #[test] fn test_match_pattern_alternation_typechecks_and_counts_exhaustiveness() { let source = r#" @@ -3011,6 +3206,15 @@ fn test_rust_metadata_lookup_path_rejects_unknown_placeholder() { assert_eq!(TypeChecker::rust_metadata_lookup_path("{unknown}"), None); } +#[test] +fn test_rust_display_unknown_placeholder_resolves_unknown() { + let checker = TypeChecker::new(); + assert_eq!( + checker.resolved_type_from_rust_display("{unknown}"), + ResolvedType::Unknown + ); +} + #[cfg(feature = "rust_inspect")] #[test] fn test_rust_item_metadata_lookup_reuses_cached_nominal_item_for_instantiated_rust_path() @@ -3547,6 +3751,15 @@ def render[T](value: Label[T]) -> str: fn seed_async_rust_method_probe( checker: &mut TypeChecker, manifest_dir: &std::path::Path, +) -> Result<(), Box> { + seed_async_rust_method_probe_with_options_param(checker, manifest_dir, "demo::CsvReadOptions") +} + +#[cfg(feature = "rust_inspect")] +fn seed_async_rust_method_probe_with_options_param( + checker: &mut TypeChecker, + manifest_dir: &std::path::Path, + options_param_type: &str, ) -> Result<(), Box> { checker.rust_inspect_cache.insert_test_item( manifest_dir, @@ -3583,7 +3796,7 @@ fn seed_async_rust_method_probe( }, RustParam { name: Some("options".to_string()), - type_display: "demo::CsvReadOptions".to_string(), + type_display: options_param_type.to_string(), }, ], return_type: "Result<(), demo::DataFusionError>".to_string(), @@ -3682,6 +3895,38 @@ pub async def register_csv_with_await() -> None: Ok(()) } +#[cfg(feature = "rust_inspect")] +#[test] +fn test_rust_async_method_call_accepts_imported_type_with_unknown_generic_metadata() +-> Result<(), Box> { + let source = r#" +import std.async +from rust::demo import SessionContext +from rust::demo import CsvReadOptions +from rust::demo import make_context +from rust::demo import make_options + +pub async def register_csv_with_unknown_options_metadata() -> None: + ctx = make_context() + opts = make_options() + match await ctx.register_csv("orders", "orders.csv", opts): + Ok(_) => pass + Err(_) => pass +"#; + let tokens = lexer::lex(source).map_err(|errs| std::io::Error::other(format!("lex failed: {errs:?}")))?; + let ast = parser::parse(&tokens).map_err(|errs| std::io::Error::other(format!("parse failed: {errs:?}")))?; + let mut checker = TypeChecker::new(); + let tmp = seeded_rust_inspect_workspace()?; + checker.set_rust_inspect_manifest_dir(tmp.path().to_path_buf()); + seed_async_rust_method_probe_with_options_param(&mut checker, tmp.path(), "demo::CsvReadOptions")?; + checker.check_program(&ast).map_err(|errs| { + std::io::Error::other(format!( + "expected Rust async method to accept an imported Rust type when metadata has only unknown generic args: {errs:?}" + )) + })?; + Ok(()) +} + #[cfg(feature = "rust_inspect")] #[test] fn test_rust_async_method_call_without_await_is_rejected() -> Result<(), Box> { @@ -4830,6 +5075,23 @@ def main() -> int: Ok(()) } +#[test] +fn test_function_callable_name_metadata_typechecks_issue694() { + let source = r#" +def capture(func: (int) -> int) -> ((int) -> int): + name: str = func.__name__ + return func + +def registered() -> (((int) -> int) -> ((int) -> int)): + return capture + +@registered() +pub def sample(value: int) -> int: + return value + 1 +"#; + assert_check_ok(source); +} + #[test] fn test_user_defined_decorator_factory_and_stacking_apply_bottom_up() { let source = r#" @@ -4856,6 +5118,62 @@ def main() -> int: assert_check_ok(source); } +#[test] +fn test_generic_decorator_factory_with_explicit_function_type_arg_preserves_binding_type() { + let source = r#" +model ColumnExpr: + name: str + +def registered[F](name: str) -> ((F) -> F): + return (func) => func + +@registered[(str) -> ColumnExpr]("inql.functions.col") +def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) + +def main() -> ColumnExpr: + return col("id") +"#; + assert_check_ok(source); +} + +#[test] +fn test_generic_decorator_factory_infers_decorated_function_type() -> Result<(), Box> { + let source = r#" +model ColumnExpr: + name: str + +def registered[F](name: str) -> ((F) -> F): + return (func) => func + +@registered("inql.functions.col") +def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) + +def main() -> ColumnExpr: + return col("id") +"#; + let tokens = lexer::lex(source).map_err(|errs| format!("lex failed: {errs:?}"))?; + let ast = parser::parse(&tokens).map_err(|errs| format!("parse failed: {errs:?}"))?; + let mut checker = TypeChecker::new(); + checker + .check_program(&ast) + .map_err(|errs| format!("typecheck failed: {errs:?}"))?; + let symbol = checker + .lookup_symbol("col") + .ok_or_else(|| "expected decorated col binding".to_string())?; + let SymbolKind::Variable(info) = &symbol.kind else { + return Err(format!("expected decorated binding to be a value, got {:?}", symbol.kind).into()); + }; + let ResolvedType::Function(params, ret) = &info.ty else { + return Err(format!("expected decorated binding to stay callable, got {:?}", info.ty).into()); + }; + assert_eq!(params.len(), 1); + assert_eq!(params[0].ty, ResolvedType::Str); + assert_eq!(**ret, ResolvedType::Named("ColumnExpr".to_string())); + Ok(()) +} + #[test] fn test_user_defined_decorator_on_async_def_is_kept_as_candidate() { let source = r#" @@ -4980,6 +5298,24 @@ def label() -> int: .any(|err| err.message.contains("'count_factory(...)' does not return a callable")), "expected non-callable factory diagnostic, got {bad_factory:?}" ); + + let bad_result = check_str_err( + r#" +def count(func: () -> int) -> int: + return 1 + +@count +def label() -> int: + return 1 +"#, + "decorator returning non-callable should be rejected", + ); + assert!( + bad_result + .iter() + .any(|err| err.message.contains("decorator 'count' must return a callable")), + "expected non-callable decorator result diagnostic, got {bad_result:?}" + ); } #[test] @@ -5137,6 +5473,44 @@ def foo() -> Result[int, str]: assert!(result.is_err()); } +#[test] +fn test_try_requires_result_return_type() { + let source = r#" +def foo() -> int: + x: Result[int, str] = Ok(42) + return x? +"#; + let errors = check_str_err(source, "try in non-Result function should fail typechecking"); + assert!( + errors + .iter() + .any(|err| err.message.contains("enclosing function does not return Result")), + "expected non-Result enclosing function diagnostic, got {errors:?}" + ); +} + +#[test] +fn test_try_does_not_cross_closure_boundary() { + let source = r#" +def parse_value() -> Result[int, str]: + return Ok(42) + +def foo() -> Result[int, str]: + callback = () => parse_value()? + return Ok(callback()) +"#; + let errors = check_str_err( + source, + "try in closure should not target enclosing Result-returning function", + ); + assert!( + errors + .iter() + .any(|err| err.message.contains("enclosing function does not return Result")), + "expected closure boundary diagnostic, got {errors:?}" + ); +} + #[test] fn test_sleep_requires_float() { let source = r#" @@ -8054,13 +8428,14 @@ def f(w: Widget) -> None: Ok(()) } +#[cfg(feature = "rust_inspect")] #[test] fn test_rust_extension_trait_associated_call_records_param_shape() -> Result<(), Box> { let source = r#" -from rust::demo import Cursor, FileDescriptorSet, Message +from rust::demo import FileDescriptorSet, Message -def f(cursor: Cursor) -> None: - _ = FileDescriptorSet.decode(cursor) +def f(encoded: bytes) -> None: + _ = FileDescriptorSet.decode(encoded.as_slice()) "#; let tokens = lexer::lex(source).map_err(|errs| std::io::Error::other(format!("lex failed: {errs:?}")))?; let ast = parser::parse(&tokens).map_err(|errs| std::io::Error::other(format!("parse failed: {errs:?}")))?; @@ -8082,7 +8457,7 @@ def f(cursor: Cursor) -> None: signature: RustFunctionSig { params: vec![RustParam { name: Some("buf".to_string()), - type_display: "T".to_string(), + type_display: "implBuf".to_string(), }], return_type: "Self".to_string(), is_async: false, @@ -8093,31 +8468,26 @@ def f(cursor: Cursor) -> None: }, ) .map_err(|err| std::io::Error::other(format!("seed trait metadata: {err}")))?; - for path in ["demo::Cursor", "demo::FileDescriptorSet"] { - checker - .rust_inspect_cache - .insert_test_item( - &manifest_dir, - RustItemMetadata { - canonical_path: path.to_string(), - definition_path: Some(path.to_string()), - visibility: RustVisibility::Public, - kind: RustItemKind::Type(RustTypeInfo { - methods: Vec::new(), - implemented_traits: if path.ends_with("FileDescriptorSet") { - vec![RustImplementedTrait { - path: "demo::Message".to_string(), - }] - } else { - Vec::new() - }, - fields: Vec::new(), - variants: Vec::new(), - }), - }, - ) - .map_err(|err| std::io::Error::other(format!("seed type metadata: {err}")))?; - } + let path = "demo::FileDescriptorSet"; + checker + .rust_inspect_cache + .insert_test_item( + &manifest_dir, + RustItemMetadata { + canonical_path: path.to_string(), + definition_path: Some(path.to_string()), + visibility: RustVisibility::Public, + kind: RustItemKind::Type(RustTypeInfo { + methods: Vec::new(), + implemented_traits: vec![RustImplementedTrait { + path: "demo::Message".to_string(), + }], + fields: Vec::new(), + variants: Vec::new(), + }), + }, + ) + .map_err(|err| std::io::Error::other(format!("seed type metadata: {err}")))?; checker .check_program(&ast) @@ -8134,10 +8504,85 @@ def f(cursor: Cursor) -> None: .calls .call_site_callable_params .values() - .any(|params| params.len() == 1 && params[0].ty == ResolvedType::TypeVar("T".to_string())), + .any(|params| params.len() == 1 && params[0].ty == ResolvedType::TypeVar("implBuf".to_string())), "expected trait-provided decode parameter shape to be recorded, got {:?}", checker.type_info().calls.call_site_callable_params ); + assert!( + checker.type_info().rust.arg_coercions.is_empty(), + "expected trait-provided impl Trait decode to avoid borrow coercions, got {:?}", + checker.type_info().rust.arg_coercions + ); + Ok(()) +} + +#[cfg(feature = "rust_inspect")] +#[test] +fn test_rust_extension_trait_associated_call_records_param_shape_without_receiver_metadata() +-> Result<(), Box> { + let source = r#" +from rust::demo import Message +from rust::datafusion_substrait::substrait::proto import Plan as ConsumerPlan + +def f(encoded: bytes) -> None: + _ = ConsumerPlan.decode(encoded.as_slice()) +"#; + let tokens = lexer::lex(source).map_err(|errs| std::io::Error::other(format!("lex failed: {errs:?}")))?; + let ast = parser::parse(&tokens).map_err(|errs| std::io::Error::other(format!("parse failed: {errs:?}")))?; + let mut checker = TypeChecker::new(); + let tmp = seeded_rust_inspect_workspace()?; + let manifest_dir = tmp.path().to_path_buf(); + checker.set_rust_inspect_manifest_dir(manifest_dir.clone()); + checker + .rust_inspect_cache + .insert_test_item( + &manifest_dir, + RustItemMetadata { + canonical_path: "demo::Message".to_string(), + definition_path: Some("demo::Message".to_string()), + visibility: RustVisibility::Public, + kind: RustItemKind::Trait(RustTraitInfo { + items: vec![RustTraitAssoc::Function { + name: "decode".to_string(), + signature: RustFunctionSig { + params: vec![RustParam { + name: Some("buf".to_string()), + type_display: "implBuf".to_string(), + }], + return_type: "Self".to_string(), + is_async: false, + is_unsafe: false, + }, + }], + }), + }, + ) + .map_err(|err| std::io::Error::other(format!("seed trait metadata: {err}")))?; + + checker + .check_program(&ast) + .map_err(|errs| std::io::Error::other(format!("typecheck failed: {errs:?}")))?; + let uses = &checker.type_info().rust.method_trait_import_uses; + assert!( + uses.values() + .any(|import_use| import_use.binding == "Message" && import_use.method == "decode"), + "expected Message import use for unresolved receiver metadata, got {uses:?}" + ); + assert!( + checker + .type_info() + .calls + .call_site_callable_params + .values() + .any(|params| params.len() == 1 && params[0].ty == ResolvedType::TypeVar("implBuf".to_string())), + "expected trait-provided decode parameter shape without receiver metadata, got {:?}", + checker.type_info().calls.call_site_callable_params + ); + assert!( + checker.type_info().rust.arg_coercions.is_empty(), + "expected unresolved receiver trait signature to avoid borrow coercions, got {:?}", + checker.type_info().rust.arg_coercions + ); Ok(()) } @@ -9005,6 +9450,69 @@ def foo() -> str: assert!(check_str(source).is_ok()); } +#[test] +fn test_local_function_named_sleep_ms_shadows_surface_helper() { + let source = r#" +def sleep_ms(value: str) -> str: + return value + +def foo() -> str: + return sleep_ms("ok") +"#; + assert_check_ok(source); +} + +#[test] +fn test_local_function_named_some_shadows_option_constructor() { + let source = r#" +def Some(value: str) -> str: + return value + +def foo() -> str: + return Some("ok") +"#; + assert_check_ok(source); +} + +#[test] +fn test_local_function_named_list_shadows_collection_helper() { + let source = r#" +def list(value: str) -> str: + return value + +def foo() -> str: + return list("ok") +"#; + assert_check_ok(source); +} + +#[test] +fn test_decorated_function_named_sum_shadows_builtin_sum_in_inline_module_tests() { + let source = r#" +model IntExpr: + value: int + +model Measure: + kind: str + +def registered[F](function_ref: str) -> ((F) -> F): + return (func) => func + +def expr(value: int) -> IntExpr: + return IntExpr(value=value) + +@registered("demo.sum") +def sum(value: IntExpr) -> Measure: + return Measure(kind="local") + +module tests: + def test_inline_sum() -> None: + measure = sum(expr(1)) + assert measure.kind == "local" +"#; + assert_check_ok(source); +} + #[test] fn test_explicit_std_builtins_sum_call() { let source = r#" @@ -9904,10 +10412,12 @@ def main() -> None: fn test_stdlib_import_only_facades_reexport_imported_types() { let source = r#" from std.datetime.civil import Date, TimeDelta +from std.datetime.error import DateTimeError -def main() -> None: +def main() -> Result[None, DateTimeError]: renewal = Date.fromisoformat("2026-04-14")? + TimeDelta.days(30) print(renewal.isoformat()) + return Ok(None) "#; assert_check_ok(source); } @@ -10825,6 +11335,24 @@ def build() -> Widget: assert!(result.is_ok(), "expected pub import to typecheck, got: {result:?}"); } +#[test] +fn test_pub_from_import_type_alias_is_transparent() { + let source = r#" +from pub::mylib import WidgetAlias, make_widget + +def keep(widget: WidgetAlias) -> WidgetAlias: + return widget + +def build() -> WidgetAlias: + return keep(make_widget("ok")) +"#; + let result = check_str_with_library_index(source, library_index_with_mylib_exports()); + assert!( + result.is_ok(), + "expected pub-imported type alias to behave transparently, got: {result:?}" + ); +} + #[test] fn test_pub_from_import_manifest_partial_callable_typechecks() { let source = r#" @@ -10841,6 +11369,21 @@ def build() -> Widget: ); } +#[test] +fn test_pub_from_import_manifest_callable_alias_typechecks() { + let source = r#" +from pub::mylib import public_target + +def build() -> int: + return public_target(1) +"#; + let result = check_str_with_library_index(source, library_index_with_callable_alias_export()); + assert!( + result.is_ok(), + "expected pub-imported callable alias to typecheck, got: {result:?}" + ); +} + #[test] fn test_pub_imported_enum_methods_and_trait_adoption_typecheck() { let source = r#" @@ -11597,6 +12140,29 @@ def main(result: Result[int, str]) -> None: check_str(source) } +#[test] +fn test_result_unwrap_helpers_typecheck() -> Result<(), Vec> { + let source = r#" +def direct(result: Result[int, str]) -> int: + return result.unwrap() + +def fallback(result: Result[int, str]) -> int: + return result.unwrap_or(0) +"#; + + check_str(source) +} + +#[test] +fn test_option_copied_accepts_generic_reference_payloads() -> Result<(), Vec> { + let source = r#" +def copy_placeholder[T](value: Option[&T]) -> Option[T]: + return value.copied() +"#; + + check_str(source) +} + #[test] fn test_rfc070_result_combinators_reject_bad_callbacks() { let source = r#" diff --git a/src/frontend/typechecker/trait_bound_relations.rs b/src/frontend/typechecker/trait_bound_relations.rs new file mode 100644 index 000000000..eddfd8f8b --- /dev/null +++ b/src/frontend/typechecker/trait_bound_relations.rs @@ -0,0 +1,659 @@ +//! Trait-bound satisfaction and temporary capability bridges. + +use std::collections::HashMap; + +use super::TypeChecker; +use crate::frontend::resolved_type_subst::substitute_resolved_type; +use crate::frontend::symbols::{ResolvedType, TypeBoundInfo, TypeInfo}; +use crate::frontend::typechecker::helpers::collection_type_id; +use incan_core::interop::is_rust_capability_bound; +use incan_core::lang::derives::{self, DeriveId}; +use incan_core::lang::trait_capabilities::{self, TraitCapabilityInfo, TraitCapabilityType}; +use incan_core::lang::traits::{self as builtin_traits, TraitId}; +use incan_core::lang::types::collections::CollectionTypeId; +use incan_core::lang::types::numerics; + +impl TypeChecker { + /// Render a type-parameter bound with call-site substitutions applied. + pub(in crate::frontend::typechecker) fn type_bound_display( + &self, + bound: &TypeBoundInfo, + bindings: &HashMap, + ) -> String { + if bound.type_args.is_empty() { + return bound.name.clone(); + } + let args = bound + .type_args + .iter() + .map(|arg| substitute_resolved_type(arg, bindings).to_string()) + .collect::>() + .join(", "); + format!("{}[{}]", bound.name, args) + } + + /// Return whether a type satisfies one explicit bound, including generic trait arguments. + pub(crate) fn type_satisfies_explicit_bound_info( + &self, + ty: &ResolvedType, + bound: &TypeBoundInfo, + bindings: &HashMap, + ) -> bool { + if let Some(placeholder_name) = self.active_type_param_name(ty) + && self.active_type_param_satisfies_bound_info(placeholder_name, bound, bindings) + { + return true; + } + if bound.name == builtin_traits::as_str(TraitId::Awaitable) { + let expected_output = bound + .type_args + .first() + .map(|arg| substitute_resolved_type(arg, bindings)); + return self.type_satisfies_awaitable_bound(ty, expected_output.as_ref()); + } + if let Some(capability) = self.temporary_trait_capability_for_bound_info(bound) + && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) + { + return satisfies; + } + if bound.type_args.is_empty() { + return self.type_satisfies_explicit_bound(ty, &bound.name); + } + if is_rust_capability_bound(&bound.name) { + return true; + } + if builtin_traits::from_str(&bound.name).is_some() || self.lookup_semantic_trait_info(&bound.name).is_none() { + return self.type_satisfies_explicit_bound(ty, &bound.name); + } + let expected_args = bound + .type_args + .iter() + .map(|arg| substitute_resolved_type(arg, bindings)) + .collect::>(); + self.type_satisfies_nominal_trait_bound_with_args(ty, &bound.name, &expected_args) + } + + /// Best-effort check whether a concrete type satisfies an explicit generic bound. + pub(in crate::frontend::typechecker) fn type_satisfies_explicit_bound( + &self, + ty: &ResolvedType, + bound: &str, + ) -> bool { + if bound == builtin_traits::as_str(TraitId::Awaitable) { + return self.type_satisfies_awaitable_bound(ty, None); + } + if is_rust_capability_bound(bound) { + return true; + } + if let Some(capability) = self.temporary_trait_capability_for_bound(bound) + && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) + { + return satisfies; + } + if builtin_traits::from_str(bound).is_none() && self.lookup_semantic_trait_info(bound).is_some() { + return self.type_satisfies_nominal_trait_bound(ty, bound); + } + match ty { + ResolvedType::Unknown + | ResolvedType::TypeVar(_) + | ResolvedType::RustPath(_) + | ResolvedType::CallSiteInfer => true, + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Numeric(_) + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit => self.primitive_type_satisfies_bound(ty, bound), + ResolvedType::Tuple(items) => self.tuple_type_satisfies_bound(items, bound), + ResolvedType::FrozenList(inner) => self.collection_type_satisfies_bound( + CollectionTypeId::FrozenList, + std::slice::from_ref(inner.as_ref()), + bound, + ), + ResolvedType::FrozenSet(inner) => self.collection_type_satisfies_bound( + CollectionTypeId::FrozenSet, + std::slice::from_ref(inner.as_ref()), + bound, + ), + ResolvedType::FrozenDict(k, v) => { + let pair = [k.as_ref().clone(), v.as_ref().clone()]; + self.collection_type_satisfies_bound(CollectionTypeId::FrozenDict, &pair, bound) + } + ResolvedType::Generic(name, args) => { + if let Some(kind) = collection_type_id(name.as_str()) { + self.collection_type_satisfies_bound(kind, args, bound) + } else { + self.named_type_satisfies_bound(name, bound) + } + } + ResolvedType::Named(type_name) => self.named_type_satisfies_bound(type_name, bound), + ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => self.type_satisfies_explicit_bound(inner, bound), + ResolvedType::Function(_, _) | ResolvedType::SelfType => false, + } + } + + /// Return the active generic placeholder name represented by `ty`. + fn active_type_param_name<'a>(&self, ty: &'a ResolvedType) -> Option<&'a str> { + let name = match ty { + ResolvedType::TypeVar(name) | ResolvedType::Named(name) => name, + _ => return None, + }; + self.current_type_param_bound_details + .iter() + .rev() + .any(|frame| frame.contains_key(name)) + .then_some(name.as_str()) + } + + /// Check whether an active generic placeholder already carries the bound required by a nested generic call. + fn active_type_param_satisfies_bound_info( + &self, + placeholder_name: &str, + required: &TypeBoundInfo, + bindings: &HashMap, + ) -> bool { + for frame in self.current_type_param_bound_details.iter().rev() { + let Some(active_bounds) = frame.get(placeholder_name) else { + continue; + }; + for active in active_bounds { + if !Self::type_bound_names_match(active, required) { + continue; + } + if required.type_args.is_empty() { + return true; + } + if active.type_args.len() != required.type_args.len() { + continue; + } + let expected = required + .type_args + .iter() + .map(|arg| substitute_resolved_type(arg, bindings)); + let actual = active + .type_args + .iter() + .map(|arg| substitute_resolved_type(arg, bindings)); + if expected + .zip(actual) + .all(|(left, right)| self.types_compatible(&left, &right)) + { + return true; + } + } + return false; + } + false + } + + /// Return the resolved source trait item name for a bound, falling back to the visible spelling. + fn type_bound_source_name(bound: &TypeBoundInfo) -> &str { + bound + .source_name + .as_deref() + .unwrap_or_else(|| bound.name.rsplit('.').next().unwrap_or(bound.name.as_str())) + } + + /// Return whether two bound records identify the same trait, accounting for import aliases. + fn type_bound_names_match(left: &TypeBoundInfo, right: &TypeBoundInfo) -> bool { + if left.name == right.name { + return true; + } + left.module_path == right.module_path + && left.module_path.is_some() + && Self::type_bound_source_name(left) == Self::type_bound_source_name(right) + } + + /// Check whether `ty` satisfies a nominal trait bound `bound_trait` under RFC 042 semantics. + fn type_satisfies_nominal_trait_bound(&self, ty: &ResolvedType, bound_trait: &str) -> bool { + match ty { + ResolvedType::Unknown + | ResolvedType::TypeVar(_) + | ResolvedType::RustPath(_) + | ResolvedType::CallSiteInfer => true, + ResolvedType::Named(type_name) => { + if self.lookup_semantic_trait_info(type_name).is_some() { + self.trait_is_supertrait_of(type_name, bound_trait) + } else { + self.type_implements_trait(type_name, bound_trait) + } + } + ResolvedType::Generic(type_name, _args) => { + if self.lookup_semantic_trait_info(type_name).is_some() { + self.trait_is_supertrait_of(type_name, bound_trait) + } else if self.lookup_semantic_type_info(type_name).is_some() { + self.type_implements_trait(type_name, bound_trait) + } else { + false + } + } + ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { + self.type_satisfies_nominal_trait_bound(inner, bound_trait) + } + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Numeric(_) + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + | ResolvedType::Tuple(_) + | ResolvedType::FrozenList(_) + | ResolvedType::FrozenSet(_) + | ResolvedType::FrozenDict(_, _) + | ResolvedType::Function(_, _) + | ResolvedType::SelfType => false, + } + } + + /// Return whether a nominal type satisfies a trait bound with exact expected trait arguments. + fn type_satisfies_nominal_trait_bound_with_args( + &self, + ty: &ResolvedType, + bound_trait: &str, + expected_args: &[ResolvedType], + ) -> bool { + match ty { + ResolvedType::Unknown + | ResolvedType::TypeVar(_) + | ResolvedType::RustPath(_) + | ResolvedType::CallSiteInfer => true, + ResolvedType::Named(type_name) => { + self.type_implements_trait_with_args(type_name, &[], bound_trait, expected_args) + } + ResolvedType::Generic(type_name, type_args) => { + self.type_implements_trait_with_args(type_name, type_args, bound_trait, expected_args) + } + ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { + self.type_satisfies_nominal_trait_bound_with_args(inner, bound_trait, expected_args) + } + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Numeric(_) + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + | ResolvedType::Tuple(_) + | ResolvedType::FrozenList(_) + | ResolvedType::FrozenSet(_) + | ResolvedType::FrozenDict(_, _) + | ResolvedType::Function(_, _) + | ResolvedType::SelfType => false, + } + } + + /// Check a concrete model/class adoption list for a matching generic trait instantiation. + fn type_implements_trait_with_args( + &self, + type_name: &str, + concrete_type_args: &[ResolvedType], + bound_trait: &str, + expected_args: &[ResolvedType], + ) -> bool { + let Some(info) = self.lookup_semantic_type_info(type_name) else { + return false; + }; + let (owner_type_params, adoptions, derives) = match info { + TypeInfo::Model(model) => ( + model.type_params.as_slice(), + model.trait_adoptions.as_slice(), + Some(model.derives.as_slice()), + ), + TypeInfo::Class(class) => ( + class.type_params.as_slice(), + class.trait_adoptions.as_slice(), + Some(class.derives.as_slice()), + ), + TypeInfo::Enum(en) => ( + en.type_params.as_slice(), + en.trait_adoptions.as_slice(), + Some(en.derives.as_slice()), + ), + TypeInfo::Newtype(newtype) => (newtype.type_params.as_slice(), newtype.trait_adoptions.as_slice(), None), + TypeInfo::Builtin | TypeInfo::TypeAlias => return false, + }; + + if expected_args.is_empty() + && derives.is_some_and(|items| items.iter().any(|derive| derive == bound_trait)) + && self.lookup_semantic_trait_info(bound_trait).is_some() + { + return true; + } + + let owner_subst = + crate::frontend::resolved_type_subst::type_param_subst_map(owner_type_params, concrete_type_args); + for adoption in adoptions { + let Some(adopted_info) = self.lookup_semantic_trait_info(&adoption.name) else { + continue; + }; + let direct_args = if adoption.type_args.is_empty() { + concrete_type_args + .iter() + .take(adopted_info.type_params.len()) + .cloned() + .collect::>() + } else { + adoption + .type_args + .iter() + .map(|arg| substitute_resolved_type(arg, &owner_subst)) + .collect::>() + }; + if direct_args.len() != adopted_info.type_params.len() { + continue; + } + if self.trait_name_matches(&adoption.name, bound_trait) + && self.trait_args_match(&direct_args, expected_args) + { + return true; + } + + let subst = + crate::frontend::resolved_type_subst::type_param_subst_map(&adopted_info.type_params, &direct_args); + for (supertrait_name, supertrait_args) in self.semantic_supertrait_closure(&adoption.name) { + if !self.trait_name_matches(&supertrait_name, bound_trait) { + continue; + } + let instantiated = supertrait_args + .iter() + .map(|arg| substitute_resolved_type(arg, &subst)) + .collect::>(); + if self.trait_args_match(&instantiated, expected_args) { + return true; + } + } + } + false + } + + /// Compare instantiated trait arguments using the typechecker's compatibility relation. + fn trait_args_match(&self, actual_args: &[ResolvedType], expected_args: &[ResolvedType]) -> bool { + actual_args.len() == expected_args.len() + && actual_args + .iter() + .zip(expected_args.iter()) + .all(|(actual, expected)| self.types_compatible(actual, expected)) + } + + /// Return whether a primitive type satisfies a builtin or registry-backed temporary capability bound. + fn primitive_type_satisfies_bound(&self, ty: &ResolvedType, bound: &str) -> bool { + if bound == derives::as_str(DeriveId::Copy) { + return self.is_copy_type(ty); + } + if let Some(capability) = self.temporary_trait_capability_for_bound(bound) + && let Some(satisfies) = self.temporary_trait_capability_supports_type(capability, ty) + { + return satisfies; + } + + match builtin_traits::from_str(bound) { + Some(TraitId::Clone | TraitId::Debug | TraitId::Display) => matches!( + ty, + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + ), + Some(TraitId::Default) => matches!( + ty, + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + ), + Some(TraitId::Awaitable) => self.type_satisfies_awaitable_bound(ty, None), + Some(TraitId::Eq | TraitId::Ord | TraitId::Hash) => matches!( + ty, + ResolvedType::Int + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + ), + Some(TraitId::PartialEq | TraitId::PartialOrd) => matches!( + ty, + ResolvedType::Int + | ResolvedType::Float + | ResolvedType::Bool + | ResolvedType::Str + | ResolvedType::Bytes + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + ), + _ => false, + } + } + + /// Resolve a temporary trait-owned capability bridge for a bound. + fn temporary_trait_capability_for_bound(&self, bound: &str) -> Option<&'static TraitCapabilityInfo> { + let (module_path, trait_name) = self.resolve_bound_trait_path(bound)?; + let capability = trait_capabilities::for_trait_path(&module_path, &trait_name)?; + self.validated_temporary_trait_capability(capability, bound, None, None) + } + + /// Resolve a temporary capability bridge from a checked bound that may have crossed a package manifest boundary. + fn temporary_trait_capability_for_bound_info(&self, bound: &TypeBoundInfo) -> Option<&'static TraitCapabilityInfo> { + if let Some(module_path) = &bound.module_path { + let trait_name = Self::type_bound_source_name(bound); + let capability = trait_capabilities::for_trait_path(module_path, trait_name)?; + return self.validated_temporary_trait_capability( + capability, + &bound.name, + bound.source_name.as_deref(), + Some(module_path), + ); + } + self.temporary_trait_capability_for_bound(&bound.name) + } + + /// Validate that a temporary capability bridge points at a real trait with the required semantic surface. + fn validated_temporary_trait_capability( + &self, + capability: &'static TraitCapabilityInfo, + visible_bound: &str, + source_name: Option<&str>, + module_path: Option<&[String]>, + ) -> Option<&'static TraitCapabilityInfo> { + let info = self + .lookup_semantic_trait_info(visible_bound) + .or_else(|| source_name.and_then(|name| self.lookup_semantic_trait_info(name))) + .or_else(|| self.lookup_semantic_trait_info(capability.trait_name)); + if let Some(info) = info + && capability + .required_methods + .iter() + .all(|method| info.methods.contains_key(*method)) + { + return Some(capability); + } + let manifest_bound_identifies_capability = source_name == Some(capability.trait_name) + && module_path.is_some_and(|path| trait_capabilities::module_path_matches(capability, path)); + manifest_bound_identifies_capability.then_some(capability) + } + + /// Resolve a bound spelling to its defining module path and trait name. + fn resolve_bound_trait_path(&self, bound: &str) -> Option<(Vec, String)> { + if let Some(path) = self.import_aliases.get(bound) + && path.len() >= 2 + { + let trait_name = path.last()?.clone(); + let module_path = path[..path.len() - 1].to_vec(); + return Some((module_path, trait_name)); + } + if !bound.contains('.') { + let module_path = self.current_module_path.clone()?; + return Some((module_path, bound.to_string())); + } + let (module_name, trait_name) = bound.rsplit_once('.')?; + let module_path = self.module_path_for_imported_name(module_name)?; + Some((module_path, trait_name.to_string())) + } + + /// Return temporary trait satisfaction for proven source type families. + fn temporary_trait_capability_supports_type( + &self, + capability: &TraitCapabilityInfo, + ty: &ResolvedType, + ) -> Option { + match ty { + ResolvedType::Unknown + | ResolvedType::TypeVar(_) + | ResolvedType::RustPath(_) + | ResolvedType::CallSiteInfer => Some(true), + ResolvedType::Int => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Int)), + ResolvedType::Bool => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Bool)), + ResolvedType::Str => Some(trait_capabilities::supports_type(capability, TraitCapabilityType::Str)), + ResolvedType::Bytes => Some(trait_capabilities::supports_type( + capability, + TraitCapabilityType::Bytes, + )), + ResolvedType::Numeric(id) => Some(trait_capabilities::supports_type( + capability, + TraitCapabilityType::Numeric(*id), + )), + ResolvedType::Ref(inner) | ResolvedType::RefMut(inner) => { + self.temporary_trait_capability_supports_type(capability, inner) + } + ResolvedType::Generic(name, args) + if numerics::decimal_constructor_from_str(name.as_str()).is_some() + && args.len() == 2 + && args + .iter() + .all(|arg| matches!(arg, ResolvedType::TypeVar(value) if value.parse::().is_ok())) => + { + Some(trait_capabilities::supports_type( + capability, + TraitCapabilityType::Decimal, + )) + } + ResolvedType::Named(type_name) | ResolvedType::Generic(type_name, _) + if self.value_enum_type_satisfies_temporary_trait_capability(type_name) => + { + Some(trait_capabilities::supports_type( + capability, + TraitCapabilityType::ValueEnum, + )) + } + ResolvedType::Float + | ResolvedType::FrozenStr + | ResolvedType::FrozenBytes + | ResolvedType::Unit + | ResolvedType::Tuple(_) + | ResolvedType::FrozenList(_) + | ResolvedType::FrozenSet(_) + | ResolvedType::FrozenDict(_, _) + | ResolvedType::Function(_, _) + | ResolvedType::SelfType => Some(false), + ResolvedType::Generic(_, _) | ResolvedType::Named(_) => None, + } + } + + /// Return whether a nominal type is a stable scalar value enum category for temporary capability bridges. + fn value_enum_type_satisfies_temporary_trait_capability(&self, type_name: &str) -> bool { + matches!( + self.lookup_semantic_type_info(type_name), + Some(TypeInfo::Enum(info)) if info.value_enum.is_some() + ) + } + + fn tuple_type_satisfies_bound(&self, items: &[ResolvedType], bound: &str) -> bool { + match builtin_traits::from_str(bound) { + Some( + TraitId::Clone + | TraitId::Debug + | TraitId::Default + | TraitId::Eq + | TraitId::PartialEq + | TraitId::Ord + | TraitId::PartialOrd + | TraitId::Hash, + ) => items.iter().all(|item| self.type_satisfies_explicit_bound(item, bound)), + _ => false, + } + } + + fn collection_type_satisfies_bound(&self, kind: CollectionTypeId, args: &[ResolvedType], bound: &str) -> bool { + let all_args_satisfy = || args.iter().all(|arg| self.type_satisfies_explicit_bound(arg, bound)); + match builtin_traits::from_str(bound) { + Some(TraitId::Clone | TraitId::Debug) => all_args_satisfy(), + Some(TraitId::Default) => matches!( + kind, + CollectionTypeId::List + | CollectionTypeId::FrozenList + | CollectionTypeId::Dict + | CollectionTypeId::FrozenDict + | CollectionTypeId::Set + | CollectionTypeId::FrozenSet + | CollectionTypeId::Option + ), + Some(TraitId::Eq | TraitId::PartialEq) => all_args_satisfy(), + Some(TraitId::Ord | TraitId::PartialOrd) => { + matches!( + kind, + CollectionTypeId::List + | CollectionTypeId::FrozenList + | CollectionTypeId::Tuple + | CollectionTypeId::Option + ) && all_args_satisfy() + } + Some(TraitId::Hash) => { + matches!( + kind, + CollectionTypeId::List + | CollectionTypeId::FrozenList + | CollectionTypeId::Tuple + | CollectionTypeId::Option + ) && all_args_satisfy() + } + _ => false, + } + } + + /// Return whether `ty` is one of the checked await-realization paths for `Awaitable[T]`. + fn type_satisfies_awaitable_bound(&self, ty: &ResolvedType, expected_output: Option<&ResolvedType>) -> bool { + let Some(output_ty) = self.await_output_type_from_type(ty) else { + return false; + }; + expected_output.is_none_or(|expected| { + matches!(output_ty, ResolvedType::Unknown) || self.types_compatible(&output_ty, expected) + }) + } + + /// Return whether a named user type explicitly satisfies a generic trait bound. + fn named_type_satisfies_bound(&self, type_name: &str, bound: &str) -> bool { + match self.lookup_type_info(type_name) { + Some(TypeInfo::Builtin) => matches!(builtin_traits::from_str(bound), Some(TraitId::Clone | TraitId::Debug)), + Some(TypeInfo::Model(info)) => { + info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) + } + Some(TypeInfo::Class(info)) => { + info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) + } + Some(TypeInfo::Enum(info)) => { + info.traits.iter().any(|t| t == bound) || info.derives.iter().any(|d| d == bound) + } + Some(TypeInfo::Newtype(info)) => info.traits.iter().any(|t| t == bound), + Some(TypeInfo::TypeAlias) => false, + None => false, + } + } +} diff --git a/src/frontend/typechecker/type_info.rs b/src/frontend/typechecker/type_info.rs index cfc480998..7dd05d86f 100644 --- a/src/frontend/typechecker/type_info.rs +++ b/src/frontend/typechecker/type_info.rs @@ -177,6 +177,11 @@ pub struct RustInteropArtifacts { /// Declaration-level binding rewrites and visibility facts consumed by lowering. #[derive(Debug, Default, Clone)] pub struct DeclarationArtifacts { + /// Module-local function declarations keyed by source name after annotation resolution. + /// + /// Lowering consumes this instead of re-lowering raw AST annotations so aliases such as + /// `type Expr = Union[...]` do not produce a different callable surface from typechecked call sites. + pub function_bindings: HashMap, /// Module-visible static bindings keyed by local name for lowering/runtime emission. pub static_bindings: HashMap, /// Same-type method aliases keyed by nominal type name (`alias -> target_method`). @@ -427,11 +432,22 @@ pub struct StaticBindingInfo { pub is_imported: bool, } +/// Lowering metadata for one source function declaration. +#[derive(Debug, Clone, PartialEq)] +pub struct FunctionBindingInfo { + /// Typechecker-resolved source parameters, including default-presence markers. + pub params: Vec, + /// Typechecker-resolved source return type. + pub return_type: ResolvedType, +} + /// Lowering metadata for one RFC 036 decorated function binding. #[derive(Debug, Clone, PartialEq)] pub struct DecoratedFunctionBindingInfo { /// Final type of the module-visible binding after applying all user-defined decorators. pub ty: ResolvedType, + /// Original callable type before decorators are applied. + pub original_ty: ResolvedType, } /// Lowering metadata for one RFC 036 decorated method binding. diff --git a/src/frontend/vocab_ast_bridge.rs b/src/frontend/vocab_ast_bridge.rs index a752d24fb..e1664b4c1 100644 --- a/src/frontend/vocab_ast_bridge.rs +++ b/src/frontend/vocab_ast_bridge.rs @@ -880,6 +880,12 @@ fn public_scoped_symbol_call_to_internal( fn public_decorator_from_internal( decorator: &ast::Spanned, ) -> Result { + if !decorator.node.type_args.is_empty() { + return Err(VocabAstBridgeError::UnsupportedInternalExpression( + "typed decorator call-site arguments are not currently bridgeable", + )); + } + let mut args = Vec::new(); for arg in &decorator.node.args { match arg { diff --git a/src/lib.rs b/src/lib.rs index a7bdd3bdd..de6e1f5f2 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -42,4 +42,4 @@ pub use frontend::typechecker; pub use backend::IrCodegen; pub use backend::project::ProjectGenerator; -pub use format::{FormatConfig, check_formatted, format_diff, format_source, format_source_with_config}; +pub use format::{FormatConfig, FormatError, check_formatted, format_diff, format_source, format_source_with_config}; diff --git a/src/library_manifest/model.rs b/src/library_manifest/model.rs index d141d3c24..f68e1d8ce 100644 --- a/src/library_manifest/model.rs +++ b/src/library_manifest/model.rs @@ -88,6 +88,8 @@ pub struct LibraryExports { pub struct AliasExport { pub name: String, pub target_path: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub projected_function: Option, } /// Exported partial callable preset metadata. @@ -682,6 +684,7 @@ fn alias_export_from_checked(export: &CheckedAliasExport) -> AliasExport { AliasExport { name: export.name.clone(), target_path: export.target_path.clone(), + projected_function: export.projected_function.as_ref().map(function_export_from_checked), } } diff --git a/src/library_manifest/tests.rs b/src/library_manifest/tests.rs index f24963020..659660347 100644 --- a/src/library_manifest/tests.rs +++ b/src/library_manifest/tests.rs @@ -240,6 +240,67 @@ fn manifest_validation_rejects_unsupported_rust_abi_schema_version() { assert!(err.is_err(), "expected unsupported Rust ABI schema to fail"); } +#[test] +fn manifest_validation_rejects_unsupported_api_metadata_package_schema_version() { + let raw = format!( + r#"{{ + "name": "mylib", + "version": "0.1.0", + "incan_version": "{}", + "manifest_format": {}, + "exports": {{}}, + "soft_keywords": {{}}, + "contract_metadata": {{ + "api": {{ + "schema_version": {}, + "package": null, + "modules": [] + }} + }} +}}"#, + crate::version::INCAN_VERSION, + LIBRARY_MANIFEST_FORMAT, + crate::frontend::api_metadata::CHECKED_API_METADATA_SCHEMA_VERSION + 1 + ); + + let err = LibraryManifest::from_json_str(&raw); + assert!(err.is_err(), "expected unsupported API metadata schema to fail"); +} + +#[test] +fn manifest_validation_rejects_unsupported_api_metadata_module_schema_version() { + let raw = format!( + r#"{{ + "name": "mylib", + "version": "0.1.0", + "incan_version": "{}", + "manifest_format": {}, + "exports": {{}}, + "soft_keywords": {{}}, + "contract_metadata": {{ + "api": {{ + "schema_version": {}, + "package": null, + "modules": [ + {{ + "schema_version": {}, + "module_path": ["lib"], + "declarations": [] + }} + ] + }} + }} +}}"#, + crate::version::INCAN_VERSION, + LIBRARY_MANIFEST_FORMAT, + crate::frontend::api_metadata::CHECKED_API_METADATA_SCHEMA_VERSION, + crate::frontend::api_metadata::CHECKED_API_METADATA_SCHEMA_VERSION + 1 + ); + + let err = LibraryManifest::from_json_str(&raw); + assert!(err.is_err(), "expected unsupported API metadata module schema to fail"); +} + #[test] fn manifest_io_round_trip_preserves_rest_parameter_metadata() -> Result<(), Box> { let mut manifest = LibraryManifest::new("mylib", "0.1.0"); diff --git a/src/library_manifest/validation.rs b/src/library_manifest/validation.rs index 5e6eeb5f0..ee3189c3c 100644 --- a/src/library_manifest/validation.rs +++ b/src/library_manifest/validation.rs @@ -15,6 +15,7 @@ use super::{ EnumExport, EnumValueExport, EnumValueTypeExport, LIBRARY_MANIFEST_FORMAT, LibraryManifestError, ParamExport, ParamKindExport, PartialExport, RUST_ABI_SCHEMA_VERSION, VocabProviderManifest, }; +use crate::frontend::api_metadata::CHECKED_API_METADATA_SCHEMA_VERSION; use crate::frontend::contract_metadata::CONTRACT_METADATA_SCHEMA_VERSION; /// Validate one raw manifest payload before it is written or decoded into the semantic model. @@ -69,6 +70,23 @@ fn validate_contract_metadata(raw: &RawLibraryManifest) -> Result<(), LibraryMan metadata .validate() .map_err(|error| LibraryManifestError::Invalid(error.to_string()))?; + + if let Some(api) = &raw.contract_metadata.api { + if api.schema_version != CHECKED_API_METADATA_SCHEMA_VERSION { + return Err(LibraryManifestError::Invalid(format!( + "contract_metadata.api.schema_version {} is unsupported (expected {})", + api.schema_version, CHECKED_API_METADATA_SCHEMA_VERSION + ))); + } + for module in &api.modules { + if module.schema_version != CHECKED_API_METADATA_SCHEMA_VERSION { + return Err(LibraryManifestError::Invalid(format!( + "contract_metadata.api.modules schema_version {} is unsupported (expected {})", + module.schema_version, CHECKED_API_METADATA_SCHEMA_VERSION + ))); + } + } + } Ok(()) } diff --git a/src/lsp/backend.rs b/src/lsp/backend.rs index f0b6d3644..dbb7f7092 100644 --- a/src/lsp/backend.rs +++ b/src/lsp/backend.rs @@ -1054,7 +1054,7 @@ mod lsp_api_metadata_preview_tests { } #[test] - fn checked_api_previews_use_callable_rebound_function_signature() -> Result<(), String> { + fn checked_api_previews_preserve_source_signature_for_callable_rebound() -> Result<(), String> { let source = r#" pub def endpoint() -> str: return "raw" @@ -1089,8 +1089,8 @@ pub def endpoint() -> str: .ok_or_else(|| "expected checked function preview".to_string())?; assert!( - preview.markdown.contains("pub def endpoint(id: int) -> bool"), - "expected rebound callable signature in LSP preview, got:\n{}", + preview.markdown.contains("pub def endpoint() -> str"), + "expected source declaration signature in LSP preview, got:\n{}", preview.markdown ); @@ -1938,7 +1938,7 @@ fn local_signature_in_expr( }), Expr::Constructor(_, args) => local_signature_in_call_args(args, ast, source, offset), Expr::FString(parts) => parts.iter().find_map(|part| match part { - crate::frontend::ast::FStringPart::Expr(expr) => local_signature_in_expr(expr, ast, source, offset), + crate::frontend::ast::FStringPart::Expr { expr, .. } => local_signature_in_expr(expr, ast, source, offset), crate::frontend::ast::FStringPart::Literal(_) => None, }), Expr::Yield(Some(value)) => local_signature_in_expr(value, ast, source, offset), @@ -3569,7 +3569,7 @@ fn scoped_symbol_in_expr<'a>( Expr::Constructor(_, args) => scoped_symbol_in_call_args(args, ident, symbol_span, surfaces, found), Expr::FString(parts) => { for part in parts { - if let crate::frontend::ast::FStringPart::Expr(expr) = part { + if let crate::frontend::ast::FStringPart::Expr { expr, .. } = part { scoped_symbol_in_expr(expr, ident, symbol_span, surfaces, found); } } @@ -4086,7 +4086,7 @@ fn scoped_symbol_context_in_expr(expr: &Spanned, offset: usize, context: & Expr::Constructor(_, args) => scoped_symbol_context_in_call_args(args, offset, context), Expr::FString(parts) => { for part in parts { - if let crate::frontend::ast::FStringPart::Expr(expr) = part { + if let crate::frontend::ast::FStringPart::Expr { expr, .. } = part { scoped_symbol_context_in_expr(expr, offset, context); } } diff --git a/src/lsp/call_site_type_args.rs b/src/lsp/call_site_type_args.rs index 3d307022e..64331748e 100644 --- a/src/lsp/call_site_type_args.rs +++ b/src/lsp/call_site_type_args.rs @@ -251,8 +251,8 @@ fn call_site_type_in_expr(expr: &Spanned, offset: usize) -> Option<&Spanne Expr::Paren(inner) => call_site_type_in_expr(inner, offset), Expr::Constructor(_, args) => scan_call_args(args, offset), Expr::FString(parts) => parts.iter().find_map(|p| { - if let crate::frontend::ast::FStringPart::Expr(e) = p { - call_site_type_in_expr(e, offset) + if let crate::frontend::ast::FStringPart::Expr { expr, .. } = p { + call_site_type_in_expr(expr, offset) } else { None } diff --git a/tests/cli_integration.rs b/tests/cli_integration.rs index 80b82d3da..271743cd0 100644 --- a/tests/cli_integration.rs +++ b/tests/cli_integration.rs @@ -24,6 +24,10 @@ fn run_incan(current_dir: &Path, args: &[&str]) -> Result Result<(), Box None: + println(str(value())) "#, )?; @@ -607,29 +619,84 @@ edition = "2021" } #[test] -fn run_accepts_owned_incan_value_for_borrowed_generic_rust_param_issue506() -> Result<(), Box> { +fn run_accepts_generic_rust_param_scenarios_share_one_generated_project() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let main_path = write_minimal_project( tmp.path(), - "cli_borrowed_generic_rust_param_project", + "cli_generic_rust_param_scenarios", r#" [rust-dependencies] borrow_helper = { path = "rust/borrow_helper" } +decode_helper = { path = "rust/decode_helper" } +decode_trait_helper = { path = "rust/decode_trait_helper" } +prost = { path = "rust/prost" } +prost-types = { path = "rust/prost-types" } "#, )?; fs::write( &main_path, + r#"from borrowed_generic import borrowed_generic_case +from by_value_decode import by_value_decode_case +from cross_crate_decode import cross_crate_decode_case +from trait_by_value_decode import trait_by_value_decode_case + +def main() -> None: + println(borrowed_generic_case()) + println(by_value_decode_case()) + println(trait_by_value_decode_case()) + println(cross_crate_decode_case()) +"#, + )?; + fs::write( + tmp.path().join("src").join("borrowed_generic.incn"), r#"from rust::borrow_helper import takes_ref model Payload: name: str -def main() -> None: +pub def borrowed_generic_case() -> str: payload = Payload(name="demo") - println(takes_ref(payload)) + return f"borrowed:{takes_ref(payload)}" +"#, + )?; + fs::write( + tmp.path().join("src").join("by_value_decode.incn"), + r#"from rust::decode_helper import FileDescriptorSet +from rust::std::io import Cursor + +pub def by_value_decode_case() -> str: + mut cursor = Cursor.new(b"abc") + match FileDescriptorSet.decode(cursor): + Ok(_) => return "by_value:ok" + Err(_) => return "by_value:err" +"#, + )?; + fs::write( + tmp.path().join("src").join("trait_by_value_decode.incn"), + r#"from rust::decode_trait_helper import FileDescriptorSet, Message + +pub def trait_by_value_decode_case() -> str: + encoded = b"abc" + match FileDescriptorSet.decode(encoded.as_slice()): + Ok(_) => return "trait_by_value:ok" + Err(_) => return "trait_by_value:err" +"#, + )?; + fs::write( + tmp.path().join("src").join("cross_crate_decode.incn"), + r#"from rust::prost import Message +from rust::prost_types import FileDescriptorSet, ProducerPlan + +pub def cross_crate_decode_case() -> str: + producer = ProducerPlan.new() + encoded = producer.encode_to_vec() + match FileDescriptorSet.decode(encoded.as_slice()): + Ok(_) => return "cross_crate:ok" + Err(_) => return "cross_crate:err" "#, )?; + let helper_src = tmp.path().join("rust").join("borrow_helper").join("src"); fs::create_dir_all(&helper_src)?; fs::write( @@ -647,46 +714,6 @@ edition = "2021" helper_src.join("lib.rs"), "pub fn takes_ref(_value: &TValue) -> i64 { 1 }\n", )?; - - let output = run_incan( - tmp.path(), - &["run", main_path.to_str().ok_or("main path was not valid UTF-8")?], - )?; - - assert_success(&output, "incan run with borrowed generic Rust param"); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains('1'), - "expected borrowed generic Rust helper output, got:\n{stdout}" - ); - Ok(()) -} - -#[test] -fn run_accepts_by_value_generic_decode_rust_param_issue609() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_minimal_project( - tmp.path(), - "cli_by_value_generic_decode_project", - r#" - -[rust-dependencies] -decode_helper = { path = "rust/decode_helper" } -"#, - )?; - fs::write( - &main_path, - r#"from rust::decode_helper import FileDescriptorSet -from rust::std::io import Cursor - - -def main() -> None: - mut cursor = Cursor.new(b"abc") - match FileDescriptorSet.decode(cursor): - Ok(_) => println("ok") - Err(_) => println("err") -"#, - )?; let helper_src = tmp.path().join("rust").join("decode_helper").join("src"); fs::create_dir_all(&helper_src)?; fs::write( @@ -715,45 +742,6 @@ impl FileDescriptorSet { Ok(Self) } } -"#, - )?; - - let output = run_incan( - tmp.path(), - &["run", main_path.to_str().ok_or("main path was not valid UTF-8")?], - )?; - - assert_success(&output, "incan run with by-value generic decode Rust param"); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("ok"), - "expected by-value generic decode helper output, got:\n{stdout}" - ); - Ok(()) -} - -#[test] -fn run_accepts_trait_provided_by_value_generic_decode_rust_param_issue612() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_minimal_project( - tmp.path(), - "cli_trait_by_value_generic_decode_project", - r#" - -[rust-dependencies] -decode_trait_helper = { path = "rust/decode_trait_helper" } -"#, - )?; - fs::write( - &main_path, - r#"from rust::decode_trait_helper import FileDescriptorSet, Message - - -def main() -> None: - encoded = b"abc" - match FileDescriptorSet.decode(encoded.as_slice()): - Ok(_) => println("ok") - Err(_) => println("err") "#, )?; let helper_src = tmp.path().join("rust").join("decode_trait_helper").join("src"); @@ -780,11 +768,73 @@ pub struct DecodeError; pub struct FileDescriptorSet; pub trait Message: Sized { - fn decode(_buf: T) -> Result; + fn decode(_buf: impl DecodeBuf) -> Result; } impl Message for FileDescriptorSet { - fn decode(_buf: T) -> Result { + fn decode(_buf: impl DecodeBuf) -> Result { + Ok(Self) + } +} +"#, + )?; + let prost_src = tmp.path().join("rust").join("prost").join("src"); + fs::create_dir_all(&prost_src)?; + fs::write( + prost_src.parent().ok_or("prost src has no parent")?.join("Cargo.toml"), + r#"[package] +name = "prost" +version = "0.1.0" +edition = "2021" +"#, + )?; + fs::write( + prost_src.join("lib.rs"), + r#"pub trait Buf {} + +impl Buf for &[u8] {} + +pub struct DecodeError; + +pub trait Message: Sized { + fn decode(_buf: impl Buf) -> Result; +} +"#, + )?; + let prost_types_src = tmp.path().join("rust").join("prost-types").join("src"); + fs::create_dir_all(&prost_types_src)?; + fs::write( + prost_types_src + .parent() + .ok_or("prost-types src has no parent")? + .join("Cargo.toml"), + r#"[package] +name = "prost-types" +version = "0.1.0" +edition = "2021" + +[dependencies] +prost = { path = "../prost" } +"#, + )?; + fs::write( + prost_types_src.join("lib.rs"), + r#"pub struct ProducerPlan; + +impl ProducerPlan { + pub fn new() -> Self { + Self + } + + pub fn encode_to_vec(&self) -> Vec { + b"abc".to_vec() + } +} + +pub struct FileDescriptorSet; + +impl prost::Message for FileDescriptorSet { + fn decode(_buf: impl prost::Buf) -> Result { Ok(Self) } } @@ -796,14 +846,12 @@ impl Message for FileDescriptorSet { &["run", main_path.to_str().ok_or("main path was not valid UTF-8")?], )?; - assert_success( - &output, - "incan run with trait-provided by-value generic decode Rust param", - ); + assert_success(&output, "incan run with batched generic Rust param scenarios"); let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("ok"), - "expected trait-provided by-value generic decode helper output, got:\n{stdout}" + assert_eq!( + stdout.trim(), + "borrowed:1\nby_value:ok\ntrait_by_value:ok\ncross_crate:ok", + "expected batched generic Rust param output, got:\n{stdout}" ); Ok(()) } @@ -830,6 +878,14 @@ main = "src/main.incn" [rust-dependencies.serde] version = "1.0" +"#, + )?; + fs::write( + &main_path, + r#"from rust::serde import Serialize + +def main() -> None: + println("cli lifecycle ok") "#, )?; @@ -1463,6 +1519,1080 @@ pub def ping() -> str: Ok(()) } +#[test] +fn fmt_tuple_target_list_comprehension_remains_buildable() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "fmt_tuple_target_list_comp", "")?; + fs::write( + &main_path, + r#"def main() -> None: + values = ["alpha", "beta"] + labels: list[str] = [f"{idx}:{value}" for idx, value in enumerate(values)] +"#, + )?; + + let fmt_output = run_incan( + tmp.path(), + &["fmt", main_path.to_str().ok_or("main path was not valid UTF-8")?], + )?; + assert_success(&fmt_output, "incan fmt tuple-target list comprehension"); + + let formatted = fs::read_to_string(&main_path)?; + assert!( + formatted.contains("for idx, value in enumerate(values)"), + "formatter should keep tuple comprehension targets unparenthesized, got:\n{formatted}" + ); + assert!( + !formatted.contains("for (idx, value) in enumerate(values)"), + "formatter emitted parser-invalid tuple target parentheses, got:\n{formatted}" + ); + + let build_output = run_incan( + tmp.path(), + &["build", main_path.to_str().ok_or("main path was not valid UTF-8")?], + )?; + assert_success( + &build_output, + "incan build after formatting tuple-target list comprehension", + ); + Ok(()) +} + +#[test] +fn build_public_alias_of_imported_item_reexports_original_path_issue617() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "public_alias_import_reexport", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + fs::write( + src_dir.join("helper.incn"), + r#"pub def target(value: int) -> int: + """Return one incremented value.""" + return value + 1 +"#, + )?; + fs::write( + &main_path, + r#"from helper import target as target_builder + + +pub public_target = alias target_builder + + +def main() -> None: + """Exercise public alias re-export of an imported public function.""" + assert public_target(1) == 2 +"#, + )?; + + let output_dir = tmp.path().join("out"); + let build_output = run_incan( + tmp.path(), + &[ + "build", + main_path.to_str().ok_or("main path was not valid UTF-8")?, + output_dir.to_str().ok_or("output path was not valid UTF-8")?, + ], + )?; + assert_success(&build_output, "public alias of imported item build"); + + let generated_main = fs::read_to_string(output_dir.join("src/main.rs"))?; + assert!( + !generated_main.contains("pub use target_builder as public_target;"), + "public alias should not re-export the private local import binding, got:\n{generated_main}" + ); + assert!( + generated_main.contains("pub use crate::helper::target as public_target;") + || generated_main.contains("pub use helper::target as public_target;"), + "public alias should re-export the original imported path, got:\n{generated_main}" + ); + Ok(()) +} + +#[test] +fn build_pub_consumer_imports_public_alias_of_imported_item_issue617() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let producer_root = tmp.path().join("alias_lib"); + let producer_src = producer_root.join("src"); + fs::create_dir_all(&producer_src)?; + fs::write( + producer_root.join("incan.toml"), + r#"[project] +name = "alias_lib" +version = "0.1.0" +"#, + )?; + fs::write( + producer_src.join("helper.incn"), + r#"pub def target(value: int) -> int: + return value + 1 +"#, + )?; + fs::write( + producer_src.join("functions.incn"), + r#"from helper import target as target_impl + +pub public_target = alias target_impl +"#, + )?; + fs::write( + producer_src.join("lib.incn"), + r#"pub from functions import public_target +"#, + )?; + + let producer_build = run_incan(&producer_root, &["build", "--lib"])?; + assert_success(&producer_build, "producer build --lib for public alias issue617"); + + let manifest_path = producer_root.join("target").join("lib").join("alias_lib.incnlib"); + let manifest: serde_json::Value = serde_json::from_str(&fs::read_to_string(&manifest_path)?)?; + assert!( + manifest.pointer("/exports/aliases/0/projected_function").is_some(), + "callable alias export should include function projection metadata, got:\n{manifest}" + ); + + let consumer_root = tmp.path().join("alias_consumer"); + let consumer_main = write_minimal_project( + &consumer_root, + "alias_consumer", + r#" +[dependencies] +alias_lib = { path = "../alias_lib" } +"#, + )?; + fs::write( + &consumer_main, + r#"from pub::alias_lib import public_target + + +def main() -> None: + assert public_target(1) == 2 +"#, + )?; + + let consumer_check = run_incan( + &consumer_root, + &[ + "--check", + consumer_main.to_str().ok_or("consumer main path was not valid UTF-8")?, + ], + )?; + assert_success(&consumer_check, "pub consumer check for public alias issue617"); + Ok(()) +} + +#[test] +fn build_lib_materializes_facade_decorator_metadata_projection_issue695() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let producer_root = tmp.path().join("metadata_registry"); + let src = producer_root.join("src"); + let operators = src.join("functions").join("operators"); + fs::create_dir_all(&operators)?; + fs::write( + producer_root.join("incan.toml"), + r#"[project] +name = "metadata_registry" +version = "0.1.0" +"#, + )?; + fs::write( + src.join("registry.incn"), + r#"pub def registered[F](spec: str) -> ((F) -> F): + return (func) => func +"#, + )?; + fs::write( + operators.join("eq.incn"), + r#"from registry import registered + +pub model ColumnExpr: + pub name: str + +@registered("equal") +pub def eq(left: ColumnExpr, right: ColumnExpr) -> ColumnExpr: + return left +"#, + )?; + fs::write( + operators.join("mod.incn"), + "pub from functions.operators.eq import eq\n", + )?; + fs::write(src.join("lib.incn"), "pub from functions.operators.mod import eq\n")?; + + let producer_build = run_incan(&producer_root, &["build", "--lib"])?; + assert_success( + &producer_build, + "producer build --lib for decorator metadata projection issue695", + ); + + let manifest_path = producer_root + .join("target") + .join("lib") + .join("metadata_registry.incnlib"); + let manifest: serde_json::Value = serde_json::from_str(&fs::read_to_string(&manifest_path)?)?; + assert!( + manifest.pointer("/exports/aliases/0/projected_function").is_some(), + "reexport-only facade should materialize callable alias projection in manifest exports, got:\n{manifest}" + ); + let api_modules = manifest + .pointer("/contract_metadata/api/modules") + .and_then(|value| value.as_array()) + .ok_or("expected checked API modules in manifest")?; + let lib_alias = api_modules + .iter() + .flat_map(|module| { + module + .pointer("/declarations") + .and_then(|value| value.as_array()) + .into_iter() + .flatten() + }) + .find(|decl| { + decl.pointer("/kind").and_then(|value| value.as_str()) == Some("alias") + && decl.pointer("/name").and_then(|value| value.as_str()) == Some("eq") + && decl.pointer("/projected_function").is_some() + }) + .ok_or("expected projected eq alias declaration in checked API metadata")?; + assert_eq!( + lib_alias + .pointer("/projected_function/callable/name") + .and_then(|value| value.as_str()), + Some("eq") + ); + assert_eq!( + lib_alias + .pointer("/projected_function/source_path") + .and_then(|value| value.as_array()) + .map(|values| values.iter().filter_map(|value| value.as_str()).collect::>()), + Some(vec!["functions", "operators", "eq", "eq"]) + ); + assert!( + lib_alias + .pointer("/projected_function/decorators/0/decorated_callable/name") + .and_then(|value| value.as_str()) + == Some("eq"), + "projected decorator metadata should carry decorated callable identity/signature, got:\n{lib_alias}" + ); + Ok(()) +} + +#[test] +fn test_accepts_public_alias_of_imported_item_issue631() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "public_alias_test_reexport", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("helper.incn"), + r#"pub def target() -> int: + return 1 +"#, + )?; + fs::write( + src_dir.join("functions.incn"), + r#"from helper import target as target_builder + +pub public_target = alias target_builder +"#, + )?; + fs::write( + &main_path, + r#"from functions import public_target + + +def main() -> None: + assert public_target() == 1 +"#, + )?; + fs::write( + tests_dir.join("test_alias.incn"), + r#"from functions import public_target + + +def test_alias() -> None: + assert public_target() == 1 +"#, + )?; + + let build_output = run_incan( + tmp.path(), + &["build", main_path.to_str().ok_or("main path was not valid UTF-8")?], + )?; + assert_success(&build_output, "incan build for public alias issue631"); + + let test_path = tests_dir.join("test_alias.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for public alias issue631"); + Ok(()) +} + +#[test] +fn test_imported_public_partial_presets_keep_projected_call_surface_issue698() -> Result<(), Box> +{ + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "imported_public_partial_preset", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("presets.incn"), + r#"pub model Spec: + pub namespace: str + pub policy: str + pub klass: str + pub lifecycle: str + + +"""Build a core portable spec.""" +pub core_spec = partial Spec(namespace="core", policy="portable") +"#, + )?; + fs::write( + tests_dir.join("test_imported_partial.incn"), + r#"from presets import core_spec + + +def test_imported_partial_preset_keeps_presets() -> None: + spec = core_spec(klass="scalar", lifecycle="v1") + assert spec.namespace == "core" + assert spec.policy == "portable" + assert spec.klass == "scalar" + assert spec.lifecycle == "v1" +"#, + )?; + + let test_path = tests_dir.join("test_imported_partial.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for imported public partial issue698"); + Ok(()) +} + +#[test] +fn test_imported_partial_preset_defaults_survive_decorator_argument_issue698() -> Result<(), Box> +{ + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "imported_partial_decorator_argument", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("function_registry.incn"), + r#"pub model FunctionSpec: + pub namespace: str + pub deterministic: bool + pub lifecycle: str + + +pub static registered_names: list[str] = [] +pub static registered_namespaces: list[str] = [] + + +pub def capture(func: (int) -> int) -> ((int) -> int): + registered_names.append(func.__name__) + return func + + +pub def add(spec: FunctionSpec) -> (((int) -> int) -> ((int) -> int)): + registered_namespaces.append(spec.namespace) + return capture + + +pub deterministic_spec = partial FunctionSpec(namespace="core", deterministic=true) +"#, + )?; + fs::write( + src_dir.join("helpers.incn"), + r#"from function_registry import add, deterministic_spec + + +@add(deterministic_spec(lifecycle="stable")) +pub def normalize(value: int) -> int: + return value +"#, + )?; + fs::write( + src_dir.join("registry_facade.incn"), + r#"pub from function_registry import add, deterministic_spec +"#, + )?; + fs::write( + src_dir.join("facade_helpers.incn"), + r#"from registry_facade import add, deterministic_spec + + +@add(deterministic_spec(lifecycle="stable")) +pub def facade_normalize(value: int) -> int: + return value +"#, + )?; + fs::write( + tests_dir.join("test_registry_intent.incn"), + r#"from function_registry import registered_names, registered_namespaces +from helpers import normalize +from facade_helpers import facade_normalize + + +def test_decorator_can_infer_name_with_imported_partial_spec() -> None: + assert normalize(7) == 7 + assert registered_names[0] == "normalize" + assert registered_namespaces[0] == "core" + + +def test_decorator_can_use_reexported_partial_spec() -> None: + assert facade_normalize(8) == 8 + assert registered_names[1] == "facade_normalize" + assert registered_namespaces[1] == "core" +"#, + )?; + + let test_path = tests_dir.join("test_registry_intent.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success( + &test_output, + "incan test for imported partial in decorator argument issue698", + ); + Ok(()) +} + +#[test] +fn test_imported_partial_default_symbols_survive_decorator_argument_issue701() -> Result<(), Box> +{ + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "imported_partial_default_symbols_decorator", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("registry.incn"), + r#"pub const DEFAULT_NAMESPACE: str = "core" + + +pub enum Policy(str): + Portable = "portable" + + +pub model Spec: + pub namespace: str + pub policy: Policy + pub lifecycle: str + + +pub static namespaces: list[str] = [] +pub static names: list[str] = [] + + +pub spec = partial Spec(namespace=DEFAULT_NAMESPACE, policy=Policy.Portable) + + +pub def capture(func: (int) -> int) -> ((int) -> int): + names.append(func.__name__) + return func + + +pub def add(spec_value: Spec) -> (((int) -> int) -> ((int) -> int)): + namespaces.append(spec_value.namespace) + return capture +"#, + )?; + fs::write( + src_dir.join("helpers.incn"), + r#"from registry import add, spec + + +@add(spec(lifecycle="v1")) +pub def sample(value: int) -> int: + return value + 1 +"#, + )?; + fs::write( + tests_dir.join("test_partial_default_symbols.incn"), + r#"from helpers import sample +from registry import names, namespaces + + +def test_partial_default_symbols_in_decorator() -> None: + assert sample(1) == 2 + assert names[0] == "sample" + assert namespaces[0] == "core" +"#, + )?; + + let test_path = tests_dir.join("test_partial_default_symbols.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for imported partial default symbols issue701"); + Ok(()) +} + +#[test] +fn test_decorated_functions_preserve_default_argument_calls_issue703() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "decorated_default_argument_calls", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + fs::write( + src_dir.join("columns.incn"), + r#"pub model ColumnExpr: + pub value: str + + +pub model Ref: + pub name: str + + +pub model Literal: + pub value: int + + +pub type Expr = Union[Ref, Literal] + + +pub def col(value: str) -> ColumnExpr: + return ColumnExpr(value=value) + + +pub def union_col(name: str) -> Expr: + return Ref(name=name) +"#, + )?; + fs::write( + src_dir.join("defaults.incn"), + r#"pub model Ref: + pub name: str + + +pub model Literal: + pub value: int + + +pub type Expr = Union[Ref, Literal] + + +pub def col(name: str) -> Expr: + return Ref(name=name) + + +def identity(func: (Expr) -> int) -> (Expr) -> int: + return func + + +@identity +pub def decorated_default(expr: Expr = col("")) -> int: + return 1 +"#, + )?; + fs::write( + src_dir.join("test_consumer.incn"), + r#"from defaults import decorated_default + + +def test_imported_decorated_default_call() -> None: + assert decorated_default() == 1 +"#, + )?; + fs::write( + src_dir.join("facade.incn"), + r#"pub from defaults import decorated_default +"#, + )?; + fs::write( + src_dir.join("facade_chain.incn"), + r#"pub from facade import decorated_default +"#, + )?; + fs::write( + src_dir.join("facade_alias.incn"), + r#"pub from defaults import decorated_default as public_decorated_default +"#, + )?; + fs::write( + src_dir.join("test_facade_consumer.incn"), + r#"from facade import decorated_default + + +def test_reexported_decorated_default_call() -> None: + assert decorated_default() == 1 +"#, + )?; + fs::write( + src_dir.join("test_facade_chain_consumer.incn"), + r#"from facade_chain import decorated_default + + +def test_chained_reexported_decorated_default_call() -> None: + assert decorated_default() == 1 +"#, + )?; + fs::write( + src_dir.join("test_facade_alias_consumer.incn"), + r#"from facade_alias import public_decorated_default + + +def test_aliased_reexported_decorated_default_call() -> None: + assert public_decorated_default() == 1 +"#, + )?; + let functions_dir = src_dir.join("functions"); + let aggregates_dir = functions_dir.join("aggregates"); + fs::create_dir_all(&aggregates_dir)?; + fs::write( + aggregates_dir.join("count.incn"), + r#"from defaults import Expr, col + + +def identity(func: (Expr) -> int) -> (Expr) -> int: + return func + + +@identity +pub def count(expr: Expr = col("")) -> int: + return 1 +"#, + )?; + fs::write( + functions_dir.join("mod.incn"), + r#"pub from functions.aggregates.count import count +"#, + )?; + fs::write( + src_dir.join("test_nested_facade_consumer.incn"), + r#"from functions import count + + +def test_nested_reexported_decorated_default_call() -> None: + assert count() == 1 +"#, + )?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + tests_dir.join("test_decorated_default_probe.incn"), + r#"from columns import ColumnExpr, Expr, col, union_col + + +def identity(func: (int) -> int) -> ((int) -> int): + return func + + +class Box: + value: int + + @method_identity + def decorated_method_default(self, value: int = 11) -> int: + return value + + +def method_identity(func: (&Box, int) -> int) -> ((&Box, int) -> int): + return func + + +@identity +def decorated_default(value: int = 7) -> int: + return value + + +def count_identity(func: (ColumnExpr) -> int) -> ((ColumnExpr) -> int): + return func + + +@count_identity +def count(expr: ColumnExpr = col("")) -> int: + return 1 + + +def union_count_identity(func: (Expr) -> int) -> ((Expr) -> int): + return func + + +@union_count_identity +def union_count(expr: Expr = union_col("")) -> int: + return 1 + + +def adapted_impl(value: str) -> int: + return 7 + + +def string_adapter(func: (int) -> int) -> ((str) -> int): + return adapted_impl + + +@string_adapter +def surface_changed(value: int = 7) -> int: + return value + + +def plain_default(value: int = 7) -> int: + return value + + +def plain_union_default(expr: Expr = union_col("")) -> int: + return 1 + + +def test_decorated_default_probe() -> None: + assert plain_default() == 7 + assert plain_union_default() == 1 + assert plain_union_default(union_col("orders")) == 1 + assert decorated_default() == 7 + assert decorated_default(3) == 3 + box = Box(value=1) + assert box.decorated_method_default() == 11 + assert box.decorated_method_default(5) == 5 + assert count() == 1 + assert count(col("orders")) == 1 + assert union_count() == 1 + assert union_count(union_col("orders")) == 1 + assert surface_changed("changed") == 7 +"#, + )?; + + let test_path = tmp.path().join("tests/test_decorated_default_probe.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for decorated default arguments issue703"); + + let consumer_path = src_dir.join("test_consumer.incn"); + let consumer_output = run_incan( + tmp.path(), + &[ + "test", + consumer_path.to_str().ok_or("consumer path was not valid UTF-8")?, + ], + )?; + assert_success( + &consumer_output, + "incan test for imported decorated default arguments issue703", + ); + + let facade_consumer_path = src_dir.join("test_facade_consumer.incn"); + let facade_consumer_output = run_incan( + tmp.path(), + &[ + "test", + facade_consumer_path + .to_str() + .ok_or("facade consumer path was not valid UTF-8")?, + ], + )?; + assert_success( + &facade_consumer_output, + "incan test for re-exported decorated default arguments issue703", + ); + + let facade_chain_consumer_path = src_dir.join("test_facade_chain_consumer.incn"); + let facade_chain_consumer_output = run_incan( + tmp.path(), + &[ + "test", + facade_chain_consumer_path + .to_str() + .ok_or("facade chain consumer path was not valid UTF-8")?, + ], + )?; + assert_success( + &facade_chain_consumer_output, + "incan test for chained re-exported decorated default arguments issue703", + ); + + let facade_alias_consumer_path = src_dir.join("test_facade_alias_consumer.incn"); + let facade_alias_consumer_output = run_incan( + tmp.path(), + &[ + "test", + facade_alias_consumer_path + .to_str() + .ok_or("facade alias consumer path was not valid UTF-8")?, + ], + )?; + assert_success( + &facade_alias_consumer_output, + "incan test for aliased re-exported decorated default arguments issue703", + ); + + let nested_facade_consumer_path = src_dir.join("test_nested_facade_consumer.incn"); + let nested_facade_consumer_output = run_incan( + tmp.path(), + &[ + "test", + nested_facade_consumer_path + .to_str() + .ok_or("nested facade consumer path was not valid UTF-8")?, + ], + )?; + assert_success( + &nested_facade_consumer_output, + "incan test for nested re-exported decorated default arguments issue703", + ); + Ok(()) +} + +#[test] +fn test_decorator_callable_exposes_source_name_issue694() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "decorator_callable_name", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + &main_path, + r#"def main() -> None: + pass +"#, + )?; + fs::write( + src_dir.join("registry.incn"), + r#"pub static names: list[str] = [] + + +pub def capture(func: (int) -> int) -> ((int) -> int): + names.append(func.__name__) + return func + + +pub def registered() -> (((int) -> int) -> ((int) -> int)): + return capture +"#, + )?; + fs::write( + src_dir.join("registry_facade.incn"), + r#"pub from registry import names, registered +"#, + )?; + fs::write( + tests_dir.join("test_callable_name.incn"), + r#"from registry import names, registered +from registry_facade import registered as facade_registered + + +@registered() +pub def sample(value: int) -> int: + return value + 1 + + +@facade_registered() +pub def facade_sample(value: int) -> int: + return value + 2 + + +def test_decorator_can_read_specific_callable_name() -> None: + assert sample(1) == 2 + assert names[0] == "sample" + assert facade_sample(1) == 3 + assert names[1] == "facade_sample" +"#, + )?; + + let test_path = tests_dir.join("test_callable_name.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for decorator callable name issue694"); + Ok(()) +} + +#[test] +fn test_generic_decorator_callable_exposes_source_name_issue694() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "generic_decorator_callable_name", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("registry.incn"), + r#"pub static names: list[str] = [] + + +pub def capture[F](func: F) -> F: + names.append(func.__name__) + return func + + +pub def registered[F]() -> ((F) -> F): + return (func) => capture[F](func) +"#, + )?; + fs::write( + src_dir.join("helpers.incn"), + r#"from registry import names, registered + + +@registered[(int) -> int]() +pub def sample(value: int) -> int: + return value + 1 +"#, + )?; + fs::write( + tests_dir.join("test_generic_callable_name.incn"), + r#"from registry import names +from helpers import sample + + +def test_generic_decorator_can_read_callable_name() -> None: + assert sample(1) == 2 + assert names[0] == "sample" +"#, + )?; + + let test_path = tests_dir.join("test_generic_callable_name.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success(&test_output, "incan test for generic decorator callable name issue694"); + Ok(()) +} + +#[test] +fn test_generic_decorator_callable_name_accepts_imported_alias_union_issue701() -> Result<(), Box> +{ + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "generic_callable_name_imported_alias_union", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("types.incn"), + r#"pub model A: + pub value: int + + +pub model B: + pub value: int + + +pub type Expr = Union[A, B] +"#, + )?; + fs::write( + src_dir.join("registry.incn"), + r#"pub static names: list[str] = [] + + +pub def capture[F](func: F) -> F: + names.append(func.__name__) + return func + + +pub def register[F]() -> ((F) -> F): + return (func) => capture[F](func) +"#, + )?; + fs::write( + src_dir.join("helpers.incn"), + r#"from registry import register +from types import Expr + + +@register[(Expr) -> Expr]() +pub def identity_expr(value: Expr) -> Expr: + return value +"#, + )?; + fs::write( + tests_dir.join("test_alias_union_callable_name.incn"), + r#"from helpers import identity_expr +from registry import names +from types import A + + +def test_alias_union_callable_name() -> None: + identity_expr(A(value=1)) + assert names[0] == "identity_expr" +"#, + )?; + + let test_path = tests_dir.join("test_alias_union_callable_name.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success( + &test_output, + "incan test for alias/union generic callable name issue701", + ); + Ok(()) +} + +#[test] +fn test_generic_callable_name_planning_ignores_unrelated_async_signatures_issue701() +-> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let main_path = write_minimal_project(tmp.path(), "generic_callable_name_with_async_noise", "")?; + let src_dir = main_path.parent().ok_or("main path had no parent")?; + let tests_dir = tmp.path().join("tests"); + fs::create_dir_all(&tests_dir)?; + fs::write( + src_dir.join("registry.incn"), + r#"pub static names: list[str] = [] + + +pub def capture[F](func: F) -> F: + names.append(func.__name__) + return func + + +pub def register[F]() -> ((F) -> F): + return (func) => capture[F](func) +"#, + )?; + fs::write( + src_dir.join("helpers.incn"), + r#"from registry import register + + +@register[(int) -> int]() +pub def sample(value: int) -> int: + return value + 1 +"#, + )?; + fs::write( + src_dir.join("noise.incn"), + r#"pub async def unrelated_async(delay: float) -> None: + return + + +pub def unrelated_generic[T](value: T) -> T: + return value +"#, + )?; + fs::write( + tests_dir.join("test_scoped_callable_name_planning.incn"), + r#"from helpers import sample +from registry import names + + +def test_generic_callable_name_ignores_unrelated_signatures() -> None: + assert sample(1) == 2 + assert names[0] == "sample" +"#, + )?; + + let test_path = tests_dir.join("test_scoped_callable_name_planning.incn"); + let test_output = run_incan( + tmp.path(), + &["test", test_path.to_str().ok_or("test path was not valid UTF-8")?], + )?; + assert_success( + &test_output, + "incan test for scoped generic callable-name planning issue701", + ); + Ok(()) +} + #[test] fn build_frozen_uses_existing_lockfile_without_network() -> Result<(), Box> { let tmp = tempfile::tempdir()?; diff --git a/tests/codegen_snapshot_tests.rs b/tests/codegen_snapshot_tests.rs index f4b27760f..ef3235d69 100644 --- a/tests/codegen_snapshot_tests.rs +++ b/tests/codegen_snapshot_tests.rs @@ -117,6 +117,7 @@ fn generate_rust_with_widgets_manifest(source: &str) -> String { normalize_codegen_output(&code) } +#[cfg(feature = "rust_inspect")] fn generate_rust_with_substrait_probe(source: &str) -> String { let tmp = match tempfile::tempdir() { Ok(tmp) => tmp, @@ -722,6 +723,27 @@ def main(result: Result[int, str]) -> Result[int, str]: ); } +#[test] +fn test_rfc070_result_unwrap_codegen_does_not_require_debug_err() { + let source = r#" +model PlainError: + message: str + +pub def direct(result: Result[int, PlainError]) -> int: + return result.unwrap() +"#; + let rust_code = generate_rust(source); + let compact = rust_code.split_whitespace().collect::(); + assert!( + compact.contains("matchresult{Ok(__incan_ok)=>__incan_ok,Err(_)=>panic!"), + "Result.unwrap should lower to an explicit match that discards Err without a Debug bound:\n{rust_code}" + ); + assert!( + !compact.contains("result.unwrap()"), + "Result.unwrap should not lower to Rust unwrap(), which requires E: Debug:\n{rust_code}" + ); +} + #[test] fn test_rfc070_result_inspect_non_copy_observer_borrows_payload() { let source = r#" @@ -1214,6 +1236,34 @@ fn test_collections_codegen() { ); } +#[test] +fn test_issue633_question_mark_list_comprehension_codegen_uses_loop() { + let source = r#" +def parse_value(value: int) -> Result[int, str]: + return Ok(value) + + +def parse_all(values: list[int]) -> Result[list[int], str]: + return Ok([parse_value(value)? for value in values]) + + +def main() -> None: + match parse_all([1, 2, 3]): + Ok(values) => println(values[0]) + Err(err) => println(err) +"#; + let rust_code = generate_rust(source); + let compact = rust_code.split_whitespace().collect::(); + assert!( + compact.contains("letmut__incan_list=Vec::new();forvaluein(values).iter().copied(){__incan_list.push(parse_value(value)?);}__incan_list"), + "expected issue633 comprehension to lower to an outer-function loop, got:\n{rust_code}" + ); + assert!( + !compact.contains(".map(|value|parse_value(value)?)"), + "question-mark comprehension must not lower into an element-returning Rust map closure:\n{rust_code}" + ); +} + #[test] fn test_list_repeat_codegen() { let source = load_test_file("list_repeat"); @@ -2337,6 +2387,7 @@ fn test_issue217_rust_enum_match_bindings_codegen() { insta::assert_snapshot!("issue217_rust_enum_match_bindings", rust_code); } +#[cfg(feature = "rust_inspect")] #[test] fn test_issue459_rust_enum_pattern_import_codegen() { let source = load_test_file("issue459_rust_enum_pattern_import"); diff --git a/tests/fixtures/vocab_guardrails/semantic_string_audit.json b/tests/fixtures/vocab_guardrails/semantic_string_audit.json new file mode 100644 index 000000000..401b471e5 --- /dev/null +++ b/tests/fixtures/vocab_guardrails/semantic_string_audit.json @@ -0,0 +1,340 @@ +{ + "files": [ + { + "path": "crates/incan_core/src/interop/coercions.rs", + "category": "registry-backed Rust-boundary coercion policy", + "expected_count": 39, + "expected_fingerprint": "0x631ac11a12a89439" + }, + { + "path": "crates/incan_core/src/interop/extension_traits.rs", + "category": "metadata-free Rust extension-trait fallback inventory", + "expected_count": 8, + "expected_fingerprint": "0x86a8087fbbe1db3b" + }, + { + "path": "crates/incan_core/src/interop/metadata.rs", + "category": "registry-backed Rust collection metadata policy", + "expected_count": 14, + "expected_fingerprint": "0x5da489b106a16ae2" + }, + { + "path": "crates/rust_inspect/src/cache.rs", + "category": "rust-inspect cache migration compatibility", + "expected_count": 1, + "expected_fingerprint": "0x57ff753cfc22d019" + }, + { + "path": "crates/rust_inspect/src/cache_timing.rs", + "category": "rust-inspect timing environment compatibility", + "expected_count": 1, + "expected_fingerprint": "0xdc8309b97ee19675" + }, + { + "path": "crates/rust_inspect/src/extractor.rs", + "category": "rust-inspect display-shape normalization", + "expected_count": 18, + "expected_fingerprint": "0xfdfffcbe8f5fc285" + }, + { + "path": "crates/rust_inspect/src/lib.rs", + "category": "rust-inspect environment and unknown-display normalization", + "expected_count": 2, + "expected_fingerprint": "0xd1080e9767292290" + }, + { + "path": "src/backend/ir/codegen.rs", + "category": "codegen facade compatibility and Rust serde fallback", + "expected_count": 4, + "expected_fingerprint": "0xcedb21689b86932d" + }, + { + "path": "src/backend/ir/codegen/dependency_metadata.rs", + "category": "dependency metadata compatibility and web route preservation", + "expected_count": 2, + "expected_fingerprint": "0x9d1944407ac71f29" + }, + { + "path": "src/backend/ir/codegen/serde_activation.rs", + "category": "serde activation compatibility", + "expected_count": 3, + "expected_fingerprint": "0x2b22f0e7dbaebdbd" + }, + { + "path": "src/backend/ir/conversions.rs", + "category": "central argument conversion shape checks", + "expected_count": 2, + "expected_fingerprint": "0xc102811ab1ea1d38" + }, + { + "path": "src/backend/ir/emit/decls/functions.rs", + "category": "callable and Rust macro emission compatibility", + "expected_count": 2, + "expected_fingerprint": "0x99dc282326807784" + }, + { + "path": "src/backend/ir/emit/decls/impls.rs", + "category": "generated stdlib JSON/newtype helper retention", + "expected_count": 4, + "expected_fingerprint": "0x3fc0034bdf2e07be" + }, + { + "path": "src/backend/ir/emit/decls/mod.rs", + "category": "generated stdlib import path recognition", + "expected_count": 1, + "expected_fingerprint": "0x1c3e7de164e70908" + }, + { + "path": "src/backend/ir/emit/decls/structures.rs", + "category": "derive macro emission compatibility", + "expected_count": 4, + "expected_fingerprint": "0xd279ee037e30c621" + }, + { + "path": "src/backend/ir/emit/expressions/calls.rs", + "category": "testing helper and public-module emission compatibility", + "expected_count": 2, + "expected_fingerprint": "0x823a39e4d5d80958" + }, + { + "path": "src/backend/ir/emit/expressions/comprehensions.rs", + "category": "dict view compatibility emission", + "expected_count": 1, + "expected_fingerprint": "0x79dfdfd491f691d1" + }, + { + "path": "src/backend/ir/emit/expressions/indexing.rs", + "category": "callable source-name metadata emission", + "expected_count": 1, + "expected_fingerprint": "0x87dd19d1c7e84652" + }, + { + "path": "src/backend/ir/emit/expressions/methods.rs", + "category": "quarantined metadata-free method compatibility", + "expected_count": 9, + "expected_fingerprint": "0x2bda7f88a3c65087" + }, + { + "path": "src/backend/ir/emit/expressions/methods/fast_paths.rs", + "category": "registered method fast-path receiver typing", + "expected_count": 2, + "expected_fingerprint": "0xd121f2f287b25f51" + }, + { + "path": "src/backend/ir/emit/expressions/mod.rs", + "category": "numeric emission compatibility", + "expected_count": 1, + "expected_fingerprint": "0x7ab516025dc0a176" + }, + { + "path": "src/backend/ir/emit/mod.rs", + "category": "Rust path segment escaping compatibility", + "expected_count": 1, + "expected_fingerprint": "0x90822765d714d957" + }, + { + "path": "src/backend/ir/emit/program.rs", + "category": "callable source-name metadata use analysis", + "expected_count": 2, + "expected_fingerprint": "0x497886b639dd72c9" + }, + { + "path": "src/backend/ir/emit/types.rs", + "category": "Rust path and static trait emission compatibility", + "expected_count": 2, + "expected_fingerprint": "0x5656f96237432bea" + }, + { + "path": "src/backend/ir/expr.rs", + "category": "known iterator method enum classification", + "expected_count": 23, + "expected_fingerprint": "0x5c7ee976092c9c9a" + }, + { + "path": "src/backend/ir/lower/decl/functions.rs", + "category": "generic callable source-name lowering compatibility", + "expected_count": 2, + "expected_fingerprint": "0xc94cea987d050f33" + }, + { + "path": "src/backend/ir/lower/decl/helpers.rs", + "category": "primitive, derive, and Rust namespace lowering compatibility", + "expected_count": 10, + "expected_fingerprint": "0xb30118abb73ddcee" + }, + { + "path": "src/backend/ir/lower/decl/methods.rs", + "category": "callable, JSON, and iterator method lowering compatibility", + "expected_count": 4, + "expected_fingerprint": "0xbcf2d12b24fabc65" + }, + { + "path": "src/backend/ir/lower/decl/mod.rs", + "category": "derive const lowering compatibility", + "expected_count": 1, + "expected_fingerprint": "0x41b88bc9914484bf" + }, + { + "path": "src/backend/ir/lower/decl/traits.rs", + "category": "iterator trait method lowering compatibility", + "expected_count": 1, + "expected_fingerprint": "0x38db8c81d2ab19aa" + }, + { + "path": "src/backend/ir/lower/expr/calls.rs", + "category": "testing assert-raises lowering policy", + "expected_count": 2, + "expected_fingerprint": "0x88708e083b004857" + }, + { + "path": "src/backend/ir/lower/expr/mod.rs", + "category": "method and stdlib lowering compatibility", + "expected_count": 10, + "expected_fingerprint": "0xf0d6e2f722d351e8" + }, + { + "path": "src/backend/ir/lower/mod.rs", + "category": "newtype, derive, and validation lowering compatibility", + "expected_count": 5, + "expected_fingerprint": "0xefce60e6312a0c70" + }, + { + "path": "src/backend/ir/lower/stmt.rs", + "category": "placeholder assignment lowering compatibility", + "expected_count": 1, + "expected_fingerprint": "0xae028e0ef66ceaa2" + }, + { + "path": "src/backend/ir/lower/types.rs", + "category": "lowered primitive type spelling compatibility", + "expected_count": 6, + "expected_fingerprint": "0xa6967a320ba65785" + }, + { + "path": "src/backend/ir/reference_shape.rs", + "category": "central Rust reference-shape compatibility", + "expected_count": 1, + "expected_fingerprint": "0x22d77392903f6589" + }, + { + "path": "src/backend/ir/trait_bound_inference.rs", + "category": "clone/as_ref/self trait-bound inference compatibility", + "expected_count": 4, + "expected_fingerprint": "0x7246e6844a35b5b3" + }, + { + "path": "src/cli/commands/common.rs", + "category": "project materialization and dependency compatibility", + "expected_count": 5, + "expected_fingerprint": "0x3990288339d2b3b3" + }, + { + "path": "src/dependency_resolver.rs", + "category": "dependency resolver registry and package-alias policy", + "expected_count": 18, + "expected_fingerprint": "0x12d230c26526632c" + }, + { + "path": "src/frontend/testing_markers.rs", + "category": "metadata-loaded testing marker inventory", + "expected_count": 13, + "expected_fingerprint": "0x4d4acefa2e275e14" + }, + { + "path": "src/frontend/typechecker/check_decl.rs", + "category": "declaration-level stdlib and derive compatibility", + "expected_count": 3, + "expected_fingerprint": "0xe83eca1f5db018d2" + }, + { + "path": "src/frontend/typechecker/check_expr/access.rs", + "category": "method/type access surface classification", + "expected_count": 43, + "expected_fingerprint": "0x8ea35141db935e1c" + }, + { + "path": "src/frontend/typechecker/check_expr/basics.rs", + "category": "basic expression Rust/stdlib escape compatibility", + "expected_count": 2, + "expected_fingerprint": "0xffdd8d7881dba8e2" + }, + { + "path": "src/frontend/typechecker/check_expr/calls.rs", + "category": "enum constructor member compatibility", + "expected_count": 2, + "expected_fingerprint": "0x0582b4a26a7a5b97" + }, + { + "path": "src/frontend/typechecker/check_expr/calls/constructors.rs", + "category": "constructor named-argument compatibility", + "expected_count": 3, + "expected_fingerprint": "0xa6598f835bad7c2e" + }, + { + "path": "src/frontend/typechecker/check_expr/calls/rust_boundary.rs", + "category": "Rust-boundary borrowed display recognition", + "expected_count": 2, + "expected_fingerprint": "0x2c33eb1e16ffe7d2" + }, + { + "path": "src/frontend/typechecker/check_expr/control_flow.rs", + "category": "async guard method control-flow policy", + "expected_count": 3, + "expected_fingerprint": "0x2893f90138d3e39f" + }, + { + "path": "src/frontend/typechecker/check_expr/mod.rs", + "category": "expression-level pub/rust namespace compatibility", + "expected_count": 3, + "expected_fingerprint": "0x4241a4c501ebe60c" + }, + { + "path": "src/frontend/typechecker/check_stmt.rs", + "category": "statement-level runtime exception compatibility", + "expected_count": 1, + "expected_fingerprint": "0xe98b9ff171b2c549" + }, + { + "path": "src/frontend/typechecker/collect.rs", + "category": "derive and public-module collection compatibility", + "expected_count": 2, + "expected_fingerprint": "0x50547807f8d5d3bb" + }, + { + "path": "src/frontend/typechecker/collect/decorators.rs", + "category": "decorator and Rust module collection compatibility", + "expected_count": 3, + "expected_fingerprint": "0x380186b8c04e5753" + }, + { + "path": "src/frontend/typechecker/collect/stdlib_imports.rs", + "category": "stdlib import and extension-trait compatibility", + "expected_count": 8, + "expected_fingerprint": "0x3dbd625f6b96862d" + }, + { + "path": "src/frontend/typechecker/const_eval.rs", + "category": "Rust module const classification compatibility", + "expected_count": 1, + "expected_fingerprint": "0x654cd60a41fc2f2f" + }, + { + "path": "src/frontend/typechecker/mod.rs", + "category": "Rust display type parsing and stdlib derive compatibility", + "expected_count": 34, + "expected_fingerprint": "0xa66345b30dd9c215" + }, + { + "path": "src/frontend/typechecker/stdlib_loader.rs", + "category": "stdlib loader primitive compatibility", + "expected_count": 6, + "expected_fingerprint": "0x12908cd63ee9cda3" + }, + { + "path": "src/frontend/typechecker/validate_rust_module.rs", + "category": "Rust module validation compatibility", + "expected_count": 1, + "expected_fingerprint": "0x3814274efdb9fae2" + } + ] +} diff --git a/tests/generated_rust_artifact_tests.rs b/tests/generated_rust_artifact_tests.rs index 9619d7856..fee173405 100644 --- a/tests/generated_rust_artifact_tests.rs +++ b/tests/generated_rust_artifact_tests.rs @@ -28,6 +28,10 @@ fn run_incan(current_dir: &Path, args: &[&str]) -> Result Result<(), Box Result<(), Box> { +fn generated_library_and_pub_dependency_consumer_artifacts_match_baseline() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let project_root = tmp.path().join("artifact_widgets_project"); let src_dir = project_root.join("src"); @@ -204,25 +208,6 @@ fn generated_library_artifact_matches_baseline() -> Result<(), Box Result<(), Box> { - let tmp = tempfile::tempdir()?; - let producer_root = tmp.path().join("artifact_widgets_project"); - let producer_src = producer_root.join("src"); - fs::create_dir_all(&producer_src)?; - fs::write( - producer_root.join("incan.toml"), - "[project]\nname = \"artifact_widgets_core\"\nversion = \"0.1.0\"\n", - )?; - write_fixture(&producer_src.join("widgets.incn"), "library_widgets.incn")?; - write_fixture(&producer_src.join("lib.incn"), "library_lib.incn")?; - - let producer_build = run_incan(&producer_root, &["build", "--lib"])?; - assert_success(&producer_build, "incan build --lib producer artifact"); - let consumer_root = tmp.path().join("artifact_consumer_project"); let consumer_src = consumer_root.join("src"); fs::create_dir_all(&consumer_src)?; diff --git a/tests/generated_rust_callability_artifact_tests.rs b/tests/generated_rust_callability_artifact_tests.rs index 8efe1ad67..47ecf67e8 100644 --- a/tests/generated_rust_callability_artifact_tests.rs +++ b/tests/generated_rust_callability_artifact_tests.rs @@ -23,6 +23,10 @@ fn run_incan(current_dir: &Path, args: &[&str]) -> Result String { - let mut out = String::with_capacity(text.len()); - let mut chars = text.chars().peekable(); - while let Some(ch) = chars.next() { - if ch == '\u{1b}' && chars.peek() == Some(&'[') { - let _ = chars.next(); - for c in chars.by_ref() { - if c == 'm' { - break; - } - } - continue; - } - out.push(ch); - } - out -} - fn write_fixture_file(root: &Path, relative_path: &str, contents: &str) -> Result<(), Box> { let path = root.join(relative_path); if let Some(parent) = path.parent() { @@ -133,7 +110,7 @@ fn function_param_ty<'a>( } #[test] -fn build_lib_emits_package_facing_callable_artifact_layout() -> Result<(), Box> { +fn generated_callable_artifact_and_consumers_share_producer_build() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let producer = build_producer(tmp.path())?; let artifact = producer.join("target/lib"); @@ -178,26 +155,19 @@ fn build_lib_emits_package_facing_callable_artifact_layout() -> Result<(), Box Result<(), Box> -{ - let tmp = tempfile::tempdir()?; - build_producer(tmp.path())?; - let (consumer, main_path) = write_consumer( + let (owned_consumer, owned_main_path) = write_consumer( tmp.path(), "owned_consumer", include_str!("fixtures/generated_rust_callability/consumer_owned/src/main.incn"), )?; - let out_dir = consumer.join("out"); + let out_dir = owned_consumer.join("out"); let build_output = run_incan( - &consumer, + &owned_consumer, &[ "build", - main_path.to_str().ok_or("main path was not valid UTF-8")?, + owned_main_path.to_str().ok_or("main path was not valid UTF-8")?, out_dir.to_str().ok_or("out path was not valid UTF-8")?, ], )?; @@ -218,48 +188,5 @@ fn consumer_can_call_owned_callable_export_across_generated_package_boundary() - "expected final generated Rust project to call imported callable export, got:\n{generated_main}" ); - let run_output = run_incan( - &consumer, - &["run", main_path.to_str().ok_or("main path was not valid UTF-8")?], - )?; - assert_success(&run_output, "consumer incan run for owned callable import"); - assert_eq!(String::from_utf8_lossy(&run_output.stdout).trim(), "2\n3\n4"); - Ok(()) -} - -#[test] -fn borrowed_callable_export_is_characterized_as_current_pub_consumer_blocker() -> Result<(), Box> -{ - let tmp = tempfile::tempdir()?; - build_producer(tmp.path())?; - let (consumer, main_path) = write_consumer( - tmp.path(), - "borrowed_consumer", - include_str!("fixtures/generated_rust_callability/consumer_borrowed_blocker/src/main.incn"), - )?; - - let out_dir = consumer.join("out"); - let build_output = run_incan( - &consumer, - &[ - "build", - main_path.to_str().ok_or("main path was not valid UTF-8")?, - out_dir.to_str().ok_or("out path was not valid UTF-8")?, - ], - )?; - assert_failure(&build_output, "consumer incan build for borrowed callable import"); - - let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&build_output.stderr)); - assert!( - stderr.contains("expected fn pointer") && stderr.contains("found fn item") && stderr.contains("observe"), - "expected borrowed callable mismatch to document current pub consumer blocker, got:\n{stderr}" - ); - let generated_main = fs::read_to_string(out_dir.join("src/main.rs"))?; - assert!( - generated_main.contains("fn observe(_: Payload)") - && generated_main.contains("inspect_payload(") - && generated_main.contains(", observe)"), - "expected final generated Rust project to show consumer observer shape before Cargo type failure, got:\n{generated_main}" - ); Ok(()) } diff --git a/tests/generated_rust_native_consumer_tests.rs b/tests/generated_rust_native_consumer_tests.rs index c76bc15be..2ed82e2d2 100644 --- a/tests/generated_rust_native_consumer_tests.rs +++ b/tests/generated_rust_native_consumer_tests.rs @@ -21,6 +21,10 @@ fn run_incan(current_dir: &Path, args: &[&str]) -> Result String { out } -static RUNTIME_ERROR_PROJECT_COUNTER: AtomicU64 = AtomicU64::new(0); +/// Parse JSON log records from stdout that may also contain human logging or ordinary print lines. +fn parse_json_log_records(stdout: &str) -> Result, Box> { + stdout + .lines() + .filter(|line| line.trim_start().starts_with('{')) + .map(serde_json::from_str) + .collect::>() + .map_err(Into::into) +} -/// Create a minimal throwaway Incan project for end-to-end runtime error assertions. +/// Find a JSON logging record by its string body. +fn json_record_by_body<'a>(records: &'a [serde_json::Value], body: &str) -> Option<&'a serde_json::Value> { + records + .iter() + .find(|record| record["Body"]["StringValue"] == serde_json::json!(body)) +} + +static TEST_PROJECT_COUNTER: AtomicU64 = AtomicU64::new(0); + +/// Create a throwaway project name that does not collide under parallel nextest workers. /// -/// The generated project name includes both the current process id and a local counter so parallel nextest workers do -/// not trample each other's `target/incan/` outputs. +/// Several CLI tests rely on the default `target/incan/` output location. The generated project name includes +/// both the current process id and a local counter so those tests do not trample each other's generated Cargo projects. +fn unique_test_project_name(prefix: &str) -> String { + let unique = TEST_PROJECT_COUNTER.fetch_add(1, Ordering::Relaxed); + format!("{prefix}_{}_{}", std::process::id(), unique) +} + +/// Create a minimal throwaway Incan project for end-to-end runtime error assertions. fn write_runtime_error_project(source: &str) -> Result<(tempfile::TempDir, PathBuf), Box> { let tmp = tempfile::tempdir()?; - let unique = RUNTIME_ERROR_PROJECT_COUNTER.fetch_add(1, Ordering::Relaxed); - let project_name = format!("runtime_error_contract_{}_{}", std::process::id(), unique); + let project_name = unique_test_project_name("runtime_error_contract"); let src_dir = tmp.path().join("src"); fs::create_dir_all(&src_dir)?; fs::write( @@ -78,18 +100,7 @@ fn assert_runtime_error_cli( ) -> Result<(), Box> { let (_tmp, main_path) = write_runtime_error_project(source)?; - let check_output = Command::new(incan_debug_binary()) - .arg("--check") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - check_output.status.success(), - "expected --check to succeed so the failure is runtime.\nstderr:\n{}", - String::from_utf8_lossy(&check_output.stderr) - ); - - let run_output = Command::new(incan_debug_binary()) + let run_output = incan_command() .arg("run") .arg(&main_path) .env("CARGO_NET_OFFLINE", "true") @@ -146,7 +157,7 @@ main = "src/main.incn" "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .arg("run") .current_dir(tmp.path()) .env("CARGO_NET_OFFLINE", "true") @@ -168,10 +179,36 @@ main = "src/main.incn" } #[test] -fn std_logging_logger_surface_filters_and_preserves_bound_context() -> Result<(), Box> { - let source = r#"from std.logging import ColorPolicy, Level, LogStyle, basic_config, get_logger +fn std_logging_runtime_surfaces_share_one_generated_run() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let project_name = unique_test_project_name("std_logging_runtime_surfaces"); + let src_dir = tmp.path().join("src"); + fs::create_dir_all(&src_dir)?; + fs::write( + tmp.path().join("incan.toml"), + format!("[project]\nname = \"{project_name}\"\nversion = \"0.1.0\"\n"), + )?; + fs::write( + src_dir.join("worker.incn"), + r#"from std.logging import get_logger -def main() -> None: +pub def run_get_logger_worker() -> None: + log = get_logger() + log.info("worker ready") + +pub def run_ambient_worker() -> None: + log.info("worker ambient log ready") +"#, + )?; + let source = r#"from std.logging import ColorPolicy, Level, LogFormat, LogStyle, LoggerName, OutputTarget, basic_config, get_logger +from std.telemetry.core import TelemetryValue +from worker import run_ambient_worker, run_get_logger_worker + +model LocalLog: + def info(self, message: str) -> None: + println(f"local:{message}") + +def logger_context_case() -> None: basic_config(level=Level.WARNING, style=LogStyle.VERBOSE, color=ColorPolicy.NEVER, target="stdout") root = get_logger("app").bind({"shared": "root"}) child = root.child("loader").bind({"component": "loader"}) @@ -184,20 +221,100 @@ def main() -> None: root.error("root event") child.warning("child event", fields={"shared": "event"}) + +def json_record_shape_case() -> None: + basic_config(level=Level.DEBUG, format=LogFormat.JSON, target="stdout") + log = get_logger() + log.debug("json works", fields={"request_id": "abc", "component": "loader"}) + +def default_target_case() -> None: + basic_config(level=Level.INFO) + get_logger("app").info("stderr event") + +def shadow_case() -> None: + basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") + log = LocalLog() + log.info("shadowed") + +def ambient_root_case() -> None: + basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") + log.info("snippet ambient") + +def structured_fields_case() -> None: + basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") + log.info("structured", fields={ + "rows": 42, + "ok": true, + "ratio": 1.5, + "missing": None, + "items": TelemetryValue.array([TelemetryValue.int(1), TelemetryValue.bool(false)]), + "nested": TelemetryValue.map({"child": TelemetryValue.string("yes")}), + }) + +def telemetry_constructor_case() -> None: + text = TelemetryValue.string("alpha") + payload = TelemetryValue.map({ + "items": TelemetryValue.array([TelemetryValue.int(42), TelemetryValue.bool(true)]), + "empty": TelemetryValue.none(), + "encoded": TelemetryValue.bytes("ff"), + "ratio": TelemetryValue.float(1.5), + }) + println(f"telemetry:{text.display_text()}") + println(f"telemetry:{payload.display_text()}") + +def validator_case() -> None: + match LoggerName.from_underlying(""): + Ok(_) => println("unexpected accepted empty logger name") + Err(err) => println(f"validation:empty_logger:{err.to_string()}") + match LoggerName.from_underlying(".app"): + Ok(_) => println("unexpected accepted edge logger name") + Err(err) => println(f"validation:edge_logger:{err.to_string()}") + match LoggerName.from_underlying("app..db"): + Ok(_) => println("unexpected accepted segmented logger name") + Err(err) => println(f"validation:segmented_logger:{err.to_string()}") + match OutputTarget.from_underlying("bogus"): + Ok(_) => println("unexpected accepted output target") + Err(err) => println(f"validation:output_target:{err.to_string()}") + +def human_styles_case() -> None: + basic_config(level=Level.INFO, style=LogStyle.MINIMAL, target="stdout") + get_logger("app").info("minimal event") + basic_config(level=Level.INFO, style=LogStyle.SHORT, target="stdout") + get_logger("app").info("short event") + basic_config(level=Level.INFO, style=LogStyle.COMPLETE, target="stdout") + get_logger("app").info("complete event") + basic_config(level=Level.INFO, style=LogStyle.VERBOSE, target="stdout") + get_logger("app").info("verbose event") + run_get_logger_worker() + run_ambient_worker() + +def main() -> None: + logger_context_case() + json_record_shape_case() + default_target_case() + shadow_case() + ambient_root_case() + structured_fields_case() + telemetry_constructor_case() + validator_case() + human_styles_case() "#; + let main_path = src_dir.join("main.incn"); + fs::write(&main_path, source)?; - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) + let output = incan_command() + .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; assert!( output.status.success(), - "expected std.logging source surface run to succeed.\nstdout:\n{}\nstderr:\n{}", + "expected combined std.logging source surface run to succeed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); assert!( !stdout.contains("silent info"), "expected INFO event to be filtered by source basic_config, got:\n{stdout}" @@ -220,44 +337,34 @@ def main() -> None: stdout.contains("logger=app.loader"), "expected child logger name, got:\n{stdout}" ); - - Ok(()) -} - -#[test] -fn std_logging_source_json_renderer_preserves_record_shape() -> Result<(), Box> { - let source = r#"from std.logging import Level, LogFormat, basic_config, get_logger - -def main() -> None: - basic_config(level=Level.DEBUG, format=LogFormat.JSON, target="stdout") - log = get_logger() - log.debug("json works", fields={"request_id": "abc", "component": "loader"}) -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - output.status.success(), - "expected source-defined std.logging JSON run to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) + !stdout.contains("stderr event") && stderr.contains("stderr event"), + "expected default logging target to route the event to stderr.\nstdout:\n{stdout}\nstderr:\n{stderr}" ); - let stdout = String::from_utf8_lossy(&output.stdout); - let records: Vec = stdout - .lines() - .filter(|line| !line.trim().is_empty()) - .map(serde_json::from_str) - .collect::>()?; - assert_eq!(records.len(), 1, "expected one JSON log record, got:\n{stdout}"); - let record = &records[0]; + assert!( + stdout.contains("local:shadowed") && !stdout.contains(r#""Body":{"Type":"string","StringValue":"shadowed"}"#), + "expected local log binding to remain ordinary source, got:\n{stdout}" + ); + for expected in [ + "validation:empty_logger:std.logging logger names must not be empty", + "validation:edge_logger:std.logging logger names must not start or end with '.'", + "validation:segmented_logger:std.logging logger names must not contain empty segments", + "validation:output_target:std.logging target must be 'stdout' or 'stderr'", + ] { + assert!(stdout.contains(expected), "expected `{expected}`, got:\n{stdout}"); + } + assert!( + !stdout.contains("unexpected accepted"), + "expected std.logging validators to reject invalid values, got:\n{stdout}" + ); + + let records = parse_json_log_records(&stdout)?; + let record = json_record_by_body(&records, "json works") + .ok_or_else(|| std::io::Error::other(format!("missing `json works` record in:\n{stdout}")))?; assert_eq!(record["SeverityText"], serde_json::json!("DEBUG")); assert_eq!(record["SeverityNumber"], serde_json::json!(5)); - assert_eq!(record["InstrumentationScope"]["Name"], serde_json::json!("root")); + assert_eq!(record["InstrumentationScope"]["Name"], serde_json::json!("main")); assert_eq!(record["Body"]["Type"], serde_json::json!("string")); - assert_eq!(record["Body"]["StringValue"], serde_json::json!("json works")); assert_eq!(record["Attributes"]["request_id"]["Type"], serde_json::json!("string")); assert_eq!( record["Attributes"]["request_id"]["StringValue"], @@ -274,586 +381,180 @@ def main() -> None: "expected user fields to stay under Attributes, got:\n{record}" ); + let ambient = json_record_by_body(&records, "snippet ambient") + .ok_or_else(|| std::io::Error::other(format!("missing `snippet ambient` record in:\n{stdout}")))?; + assert_eq!(ambient["InstrumentationScope"]["Name"], serde_json::json!("main")); + + let structured = json_record_by_body(&records, "structured") + .ok_or_else(|| std::io::Error::other(format!("missing `structured` record in:\n{stdout}")))?; + let attributes = &structured["Attributes"]; + assert_eq!(attributes["rows"]["Type"], serde_json::json!("int")); + assert_eq!(attributes["rows"]["IntValue"], serde_json::json!(42)); + assert_eq!(attributes["ok"]["Type"], serde_json::json!("bool")); + assert_eq!(attributes["ok"]["BoolValue"], serde_json::json!(true)); + assert_eq!(attributes["ratio"]["Type"], serde_json::json!("float")); + assert_eq!(attributes["ratio"]["FloatValue"], serde_json::json!(1.5)); + assert_eq!(attributes["missing"]["Type"], serde_json::json!("none")); + assert_eq!(attributes["items"]["Type"], serde_json::json!("array")); + assert_eq!(attributes["nested"]["Type"], serde_json::json!("map")); + assert!( + structured.get("rows").is_none() && structured.get("nested").is_none(), + "expected structured fields to stay under Attributes, got:\n{structured}" + ); + + let log_lines: Vec<&str> = stdout.lines().filter(|line| line.contains("[INFO]")).collect(); + let short_line = log_lines + .iter() + .copied() + .find(|line| line.contains("short event")) + .unwrap_or(""); + let complete_line = log_lines + .iter() + .copied() + .find(|line| line.contains("complete event")) + .unwrap_or(""); + + assert!( + stdout.contains("[INFO] minimal event"), + "expected minimal line, got:\n{stdout}" + ); + assert_eq!( + short_line.find(" [INFO] short event"), + Some(8), + "expected short style to use compact time-of-day timestamp, got:\n{stdout}" + ); + assert!( + complete_line.contains('T') && complete_line.contains("Z [INFO] complete event"), + "expected complete style to use full datetime timestamp, got:\n{stdout}" + ); + assert!( + stdout.contains("[INFO] verbose event\n logger=app"), + "expected verbose style to add logger metadata on a second line, got:\n{stdout}" + ); + assert!( + stdout.contains("telemetry:alpha") + && stdout.contains(r#""Type":"map""#) + && stdout.contains(r#""items":{"Type":"array""#) + && stdout.contains(r#""IntValue":42"#) + && stdout.contains(r#""BoolValue":true"#) + && stdout.contains(r#""BytesValue":"ff""#) + && stdout.contains(r#""FloatValue":1.5"#), + "expected telemetry value constructors to preserve structured values, got:\n{stdout}" + ); + assert!( + stdout.contains("worker ready") + && stdout.contains("worker ambient log ready") + && stdout.contains("logger=worker") + && !stdout.contains("logger=std.logging"), + "expected worker module logging to infer logger=worker, got:\n{stdout}" + ); + Ok(()) } #[test] -fn std_logging_default_target_writes_stderr() -> Result<(), Box> { - let source = r#"from std.logging import Level, basic_config, get_logger +fn validated_newtype_runtime_scenarios() -> Result<(), Box> { + let output = incan_command() + .args([ + "run", + "-c", + r#" +type Attempts = newtype int: + def from_underlying(n: int) -> Result[Self, ValidationError]: + if n <= 0: + return Err(ValidationError("attempts must be >= 1")) + return Ok(Attempts(n)) -def main() -> None: - basic_config(level=Level.INFO) - get_logger("app").info("stderr event") -"#; +def retry(attempts: Attempts) -> None: + println(f"retry={attempts.0}") - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) +def main() -> None: + retry(3) + attempts: Attempts = 4 + println(f"local={attempts.0}") +"#, + ]) .env("CARGO_NET_OFFLINE", "true") .output()?; assert!( output.status.success(), - "expected std.logging stderr target run to succeed.\nstdout:\n{}\nstderr:\n{}", + "validated-newtype success program failed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - !stdout.contains("stderr event") && stderr.contains("stderr event"), - "expected default logging target to route the event to stderr.\nstdout:\n{stdout}\nstderr:\n{stderr}" - ); + assert!(stdout.contains("retry=3"), "unexpected stdout:\n{stdout}"); + assert!(stdout.contains("local=4"), "unexpected stdout:\n{stdout}"); + + assert_runtime_error_cli( + r#" +type Attempts = newtype int: + def from_underlying(n: int) -> Result[Self, ValidationError]: + if n <= 0: + return Err(ValidationError("attempts must be >= 1")) + return Ok(Attempts(n)) + +def retry(attempts: Attempts) -> None: + return + +def read_attempts(attempts: Attempts) -> int: + return attempts.0 + +def main() -> None: + println(f"ok={read_attempts(Attempts(1))}") + retry(0) +"#, + "ValidationError", + &["Attempts::from_underlying", "attempts must be >= 1"], + )?; + + assert_runtime_error_cli( + r#" +type PositiveInt = newtype int: + def from_underlying(n: int) -> Result[Self, ValidationError]: + if n <= 0: + return Err(ValidationError("positive int must be greater than zero")) + return Ok(PositiveInt(n)) + +model Bounds: + low: PositiveInt + high: PositiveInt + +def width(bounds: Bounds) -> int: + return bounds.high.0 - bounds.low.0 + +def main() -> None: + println(f"width={width(Bounds(low=1, high=2))}") + _ = Bounds(low=0, high=-1) +"#, + "ValidationError", + &[ + "Bounds validation failed with 2 error(s)", + "low: positive int must be greater than zero", + "high: positive int must be greater than zero", + ], + )?; Ok(()) } #[test] -fn std_logging_default_logger_infers_source_module() -> Result<(), Box> { +fn rfc028_user_defined_operators_run_end_to_end() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let src_dir = tmp.path().join("src"); fs::create_dir_all(&src_dir)?; fs::write( tmp.path().join("incan.toml"), r#"[project] -name = "std_logging_module_source" +name = "rfc028_user_defined_operators" version = "0.1.0" "#, )?; fs::write( src_dir.join("main.incn"), - r#"from std.logging import Level, LogStyle, basic_config -from worker import run_worker - -def main() -> None: - basic_config(level=Level.INFO, style=LogStyle.VERBOSE, target="stdout") - run_worker() -"#, - )?; - fs::write( - src_dir.join("worker.incn"), - r#"from std.logging import get_logger - -pub def run_worker() -> None: - log = get_logger() - log.info("worker ready") -"#, - )?; - - let output = Command::new(incan_debug_binary()) - .arg("run") - .arg("src/main.incn") - .current_dir(tmp.path()) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected module-aware std.logging run to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("worker ready") && stdout.contains("logger=worker") && !stdout.contains("logger=root"), - "expected get_logger() in worker.incn to infer logger=worker, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn std_logging_ambient_log_infers_source_module() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let src_dir = tmp.path().join("src"); - fs::create_dir_all(&src_dir)?; - fs::write( - tmp.path().join("incan.toml"), - r#"[project] -name = "std_logging_ambient_log" -version = "0.1.0" -"#, - )?; - fs::write( - src_dir.join("main.incn"), - r#"from std.logging import Level, LogStyle, basic_config -from worker import run_worker - -def main() -> None: - basic_config(level=Level.INFO, style=LogStyle.VERBOSE, target="stdout") - run_worker() -"#, - )?; - fs::write( - src_dir.join("worker.incn"), - r#"pub def run_worker() -> None: - log.info("worker ambient log ready") -"#, - )?; - - let output = Command::new(incan_debug_binary()) - .arg("run") - .arg("src/main.incn") - .current_dir(tmp.path()) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected ambient std.logging log run to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("worker ambient log ready") - && stdout.contains("logger=worker") - && !stdout.contains("logger=root") - && !stdout.contains("logger=std.logging"), - "expected ambient log in worker.incn to infer logger=worker, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn std_logging_ambient_log_is_shadowable() -> Result<(), Box> { - let source = r#"from std.logging import Level, LogFormat, basic_config - -model LocalLog: - def info(self, message: str) -> None: - println(f"local:{message}") - -def main() -> None: - basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") - log = LocalLog() - log.info("shadowed") -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected local log binding to shadow ambient std.logging.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("local:shadowed") && !stdout.contains("InstrumentationScope"), - "expected local log binding to remain ordinary source, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn std_logging_ambient_log_snippet_falls_back_to_root() -> Result<(), Box> { - let source = r#"from std.logging import Level, LogFormat, basic_config - -def main() -> None: - basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") - log.info("snippet ambient") -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected metadata-free ambient log to fall back to root.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains(r#""InstrumentationScope":{"Name":"root""#) && stdout.contains("snippet ambient"), - "expected ambient log in -c snippet to emit with root logger, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn std_logging_rejects_invalid_logger_names() -> Result<(), Box> { - let cases = [ - ( - "empty logger name", - r#"from std.logging import get_logger - -def main() -> None: - get_logger("").info("should not emit") -"#, - "std.logging logger names must not be empty", - ), - ( - "empty logger segment", - r#"from std.logging import get_logger - -def main() -> None: - get_logger("app..db").info("should not emit") -"#, - "std.logging logger names must not contain empty segments", - ), - ( - "invalid child suffix", - r#"from std.logging import get_logger - -def main() -> None: - get_logger("app").child(".db").info("should not emit") -"#, - "std.logging logger names must not contain empty segments", - ), - ]; - - for (case, source, expected) in cases { - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - !output.status.success(), - "expected {case} to fail.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let combined = format!( - "{}\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - assert!( - combined.contains(expected), - "expected {case} validation message `{expected}`, got:\n{combined}" - ); - } - - Ok(()) -} - -#[test] -fn std_logging_rejects_invalid_output_target() -> Result<(), Box> { - let source = r#"from std.logging import Level, basic_config, get_logger - -def main() -> None: - basic_config(level=Level.INFO, target="bogus") - get_logger("app").info("should not emit") -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - !output.status.success(), - "expected invalid std.logging target to fail.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let combined = format!( - "{}\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - assert!( - combined.contains("std.logging target must be 'stdout' or 'stderr'"), - "expected target validation message, got:\n{combined}" - ); - - Ok(()) -} - -#[test] -fn std_logging_json_preserves_structured_field_values() -> Result<(), Box> { - let source = r#"from std.logging import Level, LogFormat, basic_config -from std.telemetry.core import TelemetryValue - -def main() -> None: - basic_config(level=Level.INFO, format=LogFormat.JSON, target="stdout") - log.info("structured", fields={ - "rows": 42, - "ok": true, - "ratio": 1.5, - "missing": None, - "items": TelemetryValue.array([TelemetryValue.int(1), TelemetryValue.bool(false)]), - "nested": TelemetryValue.map({"child": TelemetryValue.string("yes")}), - }) -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected structured std.logging fields to compile and emit JSON.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - let records: Vec = stdout - .lines() - .filter(|line| !line.trim().is_empty()) - .map(serde_json::from_str) - .collect::>()?; - assert_eq!(records.len(), 1, "expected one JSON log record, got:\n{stdout}"); - let attributes = &records[0]["Attributes"]; - assert_eq!(attributes["rows"]["Type"], serde_json::json!("int")); - assert_eq!(attributes["rows"]["IntValue"], serde_json::json!(42)); - assert_eq!(attributes["ok"]["Type"], serde_json::json!("bool")); - assert_eq!(attributes["ok"]["BoolValue"], serde_json::json!(true)); - assert_eq!(attributes["ratio"]["Type"], serde_json::json!("float")); - assert_eq!(attributes["ratio"]["FloatValue"], serde_json::json!(1.5)); - assert_eq!(attributes["missing"]["Type"], serde_json::json!("none")); - assert_eq!(attributes["items"]["Type"], serde_json::json!("array")); - assert_eq!(attributes["nested"]["Type"], serde_json::json!("map")); - assert!( - records[0].get("rows").is_none() && records[0].get("nested").is_none(), - "expected structured fields to stay under Attributes, got:\n{}", - records[0] - ); - - Ok(()) -} - -#[test] -fn std_traits_convert_usage_runs() -> Result<(), Box> { - let source = include_str!("codegen_snapshots/std_traits_convert_usage.incn"); - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected std.traits.convert classmethod usage to compile and run.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - assert_eq!(String::from_utf8_lossy(&output.stdout), "42\n3\n"); - - Ok(()) -} - -#[test] -fn std_logging_human_styles_render_distinct_shapes() -> Result<(), Box> { - let source = r#"from std.logging import Level, LogStyle, basic_config, get_logger - -def main() -> None: - basic_config(level=Level.INFO, style=LogStyle.MINIMAL, target="stdout") - get_logger("app").info("minimal event") - basic_config(level=Level.INFO, style=LogStyle.SHORT, target="stdout") - get_logger("app").info("short event") - basic_config(level=Level.INFO, style=LogStyle.COMPLETE, target="stdout") - get_logger("app").info("complete event") - basic_config(level=Level.INFO, style=LogStyle.VERBOSE, target="stdout") - get_logger("app").info("verbose event") -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected std.logging human style run to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - let log_lines: Vec<&str> = stdout.lines().filter(|line| line.contains("[INFO]")).collect(); - let short_line = log_lines - .iter() - .copied() - .find(|line| line.contains("short event")) - .unwrap_or(""); - let complete_line = log_lines - .iter() - .copied() - .find(|line| line.contains("complete event")) - .unwrap_or(""); - - assert!( - stdout.contains("[INFO] minimal event"), - "expected minimal line, got:\n{stdout}" - ); - assert_eq!( - short_line.find(" [INFO] short event"), - Some(8), - "expected short style to use compact time-of-day timestamp, got:\n{stdout}" - ); - assert!( - complete_line.contains("T") && complete_line.contains("Z [INFO] complete event"), - "expected complete style to use full datetime timestamp, got:\n{stdout}" - ); - assert!( - stdout.contains("[INFO] verbose event\n logger=app"), - "expected verbose style to add logger metadata on a second line, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn telemetry_value_class_constructors_are_callable() -> Result<(), Box> { - let source = r#"from std.telemetry.core import TelemetryValue - -def main() -> None: - text = TelemetryValue.string("alpha") - payload = TelemetryValue.map({ - "items": TelemetryValue.array([TelemetryValue.int(42), TelemetryValue.bool(true)]), - "empty": TelemetryValue.none(), - "encoded": TelemetryValue.bytes("ff"), - "ratio": TelemetryValue.float(1.5), - }) - println(text.display_text()) - println(payload.display_text()) -"#; - - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected telemetry value constructors to run.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("alpha") - && stdout.contains(r#""Type":"map""#) - && stdout.contains(r#""items":{"Type":"array""#) - && stdout.contains(r#""IntValue":42"#) - && stdout.contains(r#""BoolValue":true"#) - && stdout.contains(r#""BytesValue":"ff""#) - && stdout.contains(r#""FloatValue":1.5"#), - "expected class constructors to preserve structured telemetry values, got:\n{stdout}" - ); - - Ok(()) -} - -#[test] -fn validated_newtype_runtime_success_coerces_approved_sites() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) - .args([ - "run", - "-c", - r#" -type Attempts = newtype int: - def from_underlying(n: int) -> Result[Self, ValidationError]: - if n <= 0: - return Err(ValidationError("attempts must be >= 1")) - return Ok(Attempts(n)) - -def retry(attempts: Attempts) -> None: - println(f"retry={attempts.0}") - -def main() -> None: - retry(3) - attempts: Attempts = 4 - println(f"local={attempts.0}") -"#, - ]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "validated-newtype success program failed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!(stdout.contains("retry=3"), "unexpected stdout:\n{stdout}"); - assert!(stdout.contains("local=4"), "unexpected stdout:\n{stdout}"); - - Ok(()) -} - -#[test] -fn validated_newtype_runtime_fail_fast_reports_validation_error() -> Result<(), Box> { - assert_runtime_error_cli( - r#" -type Attempts = newtype int: - def from_underlying(n: int) -> Result[Self, ValidationError]: - if n <= 0: - return Err(ValidationError("attempts must be >= 1")) - return Ok(Attempts(n)) - -def retry(attempts: Attempts) -> None: - return - -def read_attempts(attempts: Attempts) -> int: - return attempts.0 - -def main() -> None: - println(f"ok={read_attempts(Attempts(1))}") - retry(0) -"#, - "ValidationError", - &["Attempts::from_underlying", "attempts must be >= 1"], - ) -} - -#[test] -fn validated_newtype_runtime_aggregates_model_field_errors() -> Result<(), Box> { - assert_runtime_error_cli( - r#" -type PositiveInt = newtype int: - def from_underlying(n: int) -> Result[Self, ValidationError]: - if n <= 0: - return Err(ValidationError("positive int must be greater than zero")) - return Ok(PositiveInt(n)) - -model Bounds: - low: PositiveInt - high: PositiveInt - -def width(bounds: Bounds) -> int: - return bounds.high.0 - bounds.low.0 - -def main() -> None: - println(f"width={width(Bounds(low=1, high=2))}") - _ = Bounds(low=0, high=-1) -"#, - "ValidationError", - &[ - "Bounds validation failed with 2 error(s)", - "low: positive int must be greater than zero", - "high: positive int must be greater than zero", - ], - ) -} - -#[test] -fn rfc028_user_defined_operators_run_end_to_end() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let src_dir = tmp.path().join("src"); - fs::create_dir_all(&src_dir)?; - fs::write( - tmp.path().join("incan.toml"), - r#"[project] -name = "rfc028_user_defined_operators" -version = "0.1.0" -"#, - )?; - fs::write( - src_dir.join("main.incn"), - r#"model Money: - cents: int + r#"model Money: + cents: int def __add__(self, other: Money) -> Money: return Money(cents=self.cents + other.cents) @@ -896,7 +597,7 @@ def main() -> None: "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .arg("run") .arg("src/main.incn") .current_dir(tmp.path()) @@ -934,6 +635,38 @@ fn incan_debug_binary() -> std::path::PathBuf { std::path::PathBuf::from("target/debug/incan") } +fn shared_generated_cargo_target_dir() -> std::path::PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("incan_generated_shared_target") +} + +fn incan_command() -> Command { + let mut command = Command::new(incan_debug_binary()); + command.env("INCAN_GENERATED_CARGO_TARGET_DIR", shared_generated_cargo_target_dir()); + command +} + +fn run_incan_command_with_timeout( + mut command: Command, + timeout: std::time::Duration, +) -> std::io::Result<(std::process::Output, bool)> { + command.stdout(std::process::Stdio::piped()); + command.stderr(std::process::Stdio::piped()); + let mut child = command.spawn()?; + let start = std::time::Instant::now(); + loop { + if child.try_wait()?.is_some() { + return child.wait_with_output().map(|output| (output, false)); + } + if start.elapsed() >= timeout { + let _ = child.kill(); + return child.wait_with_output().map(|output| (output, true)); + } + std::thread::sleep(std::time::Duration::from_millis(25)); + } +} + fn is_incan_fixture(path: &Path) -> bool { matches!(path.extension().and_then(|e| e.to_str()), Some("incn") | Some("incan")) } @@ -1004,7 +737,7 @@ fn test_cli_fmt_preserves_block_decl_docstrings_and_export_doc_surface() -> Resu let dir = make_temp_test_dir(); let path = dir.join("block_docstrings_cli.incn"); fs::write(&path, BLOCK_DOCSTRING_PUBLIC_TYPE_LIKE)?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1080,7 +813,7 @@ def check_flags(ready: bool, done: bool) -> None: "#, )?; - let output = Command::new(incan_debug_binary()).arg("fmt").arg(&path).output()?; + let output = incan_command().arg("fmt").arg(&path).output()?; assert!( output.status.success(), "expected `incan fmt` to accept assert identity checks against bool literals.\nstdout:\n{}\nstderr:\n{}", @@ -1108,7 +841,7 @@ def matches(item: Item) -> bool: "#, )?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1131,7 +864,7 @@ def matches(item: Item) -> bool: "expected formatted output to stay within 120 columns:\n{formatted}" ); - let output = Command::new(incan_debug_binary()).arg("--check").arg(&path).output()?; + let output = incan_command().arg("--check").arg(&path).output()?; assert!( output.status.success(), "expected wrapped expression to parse/typecheck after CLI fmt; stderr={}", @@ -1153,7 +886,7 @@ fn test_cli_fmt_preserves_fstring_escaped_newline_roundtrip() -> Result<(), Box< "#, )?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1163,7 +896,7 @@ fn test_cli_fmt_preserves_fstring_escaped_newline_roundtrip() -> Result<(), Box< formatted ); - let output = Command::new(incan_debug_binary()).arg("--check").arg(&path).output()?; + let output = incan_command().arg("--check").arg(&path).output()?; assert!( output.status.success(), "expected formatted file to parse/typecheck after CLI fmt; stderr={}", @@ -1199,7 +932,7 @@ trait Service: "#, )?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1247,7 +980,7 @@ pub def allocate_prism_store_id() -> int: "#, )?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1282,7 +1015,7 @@ fn test_cli_fmt_keeps_trailing_comment_after_multiline_function() -> Result<(), "#, )?; - let status = Command::new(incan_debug_binary()).arg("fmt").arg(&path).status()?; + let status = incan_command().arg("fmt").arg(&path).status()?; assert!(status.success(), "incan fmt failed"); let formatted = fs::read_to_string(&path)?; @@ -1318,7 +1051,7 @@ def main() -> None: "#, )?; - let output = Command::new(incan_debug_binary()).arg("--check").arg(&path).output()?; + let output = incan_command().arg("--check").arg(&path).output()?; assert!( output.status.success(), "expected multiline trailing parameter comma to parse/typecheck; stderr={}", @@ -1403,7 +1136,7 @@ fn test_invalid_fixtures() { #[test] fn test_help_is_banner_free() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()).arg("--help").output()?; + let output = incan_command().arg("--help").output()?; assert!( output.status.success(), "incan --help failed: status={:?} stderr={}", @@ -1421,7 +1154,7 @@ fn test_help_is_banner_free() -> Result<(), Box> { #[test] fn test_version_is_single_line_and_banner_free() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()).arg("--version").output()?; + let output = incan_command().arg("--version").output()?; assert!( output.status.success(), "incan --version failed: status={:?} stderr={}", @@ -1443,7 +1176,7 @@ fn lifecycle_new_version_and_env_commands_work() -> Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box Result<(), Box None: "#, )?; - let bare_run = Command::new(incan_debug_binary()) + let bare_run = incan_command() .args(["run", "src/main.incn"]) .current_dir(project_root) .env("CARGO_NET_OFFLINE", "true") @@ -1803,7 +1536,7 @@ def main() -> None: bare_stderr ); - let env_run = Command::new(incan_debug_binary()) + let env_run = incan_command() .args(["env", "run", "unit", "run"]) .current_dir(project_root) .env("CARGO_NET_OFFLINE", "true") @@ -1856,7 +1589,7 @@ env-vars = { CHILD = "1" } "#, )?; - let bare_show = Command::new(incan_debug_binary()) + let bare_show = incan_command() .args(["env", "show", "unit", "--format", "json"]) .current_dir(project_root.join("child")) .output()?; @@ -1870,7 +1603,7 @@ env-vars = { CHILD = "1" } assert_eq!(bare_json["env_vars"]["CHILD"], "1"); assert!(bare_json["env_vars"].get("PARENT").is_none()); - let env_show = Command::new(incan_debug_binary()) + let env_show = incan_command() .args(["env", "run", "unit", "inspect"]) .current_dir(project_root) .output()?; @@ -1888,10 +1621,7 @@ env-vars = { CHILD = "1" } #[test] fn test_parse_error_is_banner_free() { - let Ok(output) = Command::new(incan_debug_binary()) - .arg("--definitely-not-a-flag") - .output() - else { + let Ok(output) = incan_command().arg("--definitely-not-a-flag").output() else { panic!("failed to run incan with invalid args"); }; assert!( @@ -1910,7 +1640,7 @@ fn test_parse_error_is_banner_free() { #[test] fn test_fstring_unknown_symbol_cli_caret_points_to_interpolation() { let source = "def main() -> str:\n return f\"value: {unknown_var}\"\n"; - let Ok(output) = Command::new(incan_debug_binary()).args(["run", "-c", source]).output() else { + let Ok(output) = incan_command().args(["run", "-c", source]).output() else { panic!("failed to run incan with f-string source"); }; @@ -1959,6 +1689,49 @@ fn test_fstring_unknown_symbol_cli_caret_points_to_interpolation() { ); } +#[test] +fn test_fstring_list_interpolation_uses_structured_formatting() -> Result<(), Box> { + let source = r#"def debug_values[T](values: list[T]) -> str: + return f"{values:?}" + +def display_values[T](values: list[T]) -> str: + return f"{values}" + +def main() -> None: + columns: list[str] = ["id", "amount"] + println(f"debug: {columns:?}") + println(f"display: {columns}") + println(debug_values[str](["id", "amount"])) + println(display_values[str](["id", "amount"])) +"#; + let output = incan_command() + .args(["run", "-c", source]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + assert!( + output.status.success(), + "expected list f-string interpolation to run.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + + let stdout = String::from_utf8_lossy(&output.stdout); + assert!( + stdout.contains("debug: [\"id\", \"amount\"]"), + "expected debug list output, got:\n{stdout}" + ); + assert!( + stdout.contains("display: [\"id\", \"amount\"]"), + "expected default list f-string output to use structured formatting, got:\n{stdout}" + ); + assert!( + stdout.lines().filter(|line| *line == "[\"id\", \"amount\"]").count() == 2, + "expected both generic list helpers to render, got:\n{stdout}" + ); + + Ok(()) +} + #[test] fn fixed_call_unpack_runs_for_positional_and_keyword_shapes() -> Result<(), Box> { let source = r#" @@ -1980,7 +1753,7 @@ def main() -> None: println(route(**{"path": "/status", "method": "GET"})) println(counter.add(*(5, 6))) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -2021,133 +1794,87 @@ def main() -> None: println(value.adjusted) println(value.label) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; - - assert!( - output.status.success(), - "expected computed property program to run.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - assert_eq!(String::from_utf8_lossy(&output.stdout), "251\nmoney\n"); - Ok(()) -} - -#[test] -fn runtime_error_missing_dict_key_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n let values = {\"a\": 1}\n println(values[\"b\"])\n", - "KeyError", - &["not found in dict"], - ) -} - -#[test] -fn runtime_error_list_index_out_of_range_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n let values = [1, 2, 3]\n println(values[99])\n", - "IndexError", - &["out of range for list"], - ) -} - -#[test] -fn runtime_error_list_index_method_not_found_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n let values = [1, 2, 3]\n println(values.index(99))\n", - "ValueError", - &["value not found in list"], - ) -} - -#[test] -fn runtime_error_int_conversion_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n println(int(\"abc\"))\n", - "ValueError", - &["cannot convert 'abc' to int"], - ) -} - -#[test] -fn runtime_error_float_conversion_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n println(float(\"abc\"))\n", - "ValueError", - &["cannot convert 'abc' to float"], - ) -} - -#[test] -fn runtime_error_list_remove_out_of_range_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n mut values = [1, 2, 3]\n values.remove(99)\n", - "IndexError", - &["out of range for list"], - ) + + assert!( + output.status.success(), + "expected computed property program to run.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + assert_eq!(String::from_utf8_lossy(&output.stdout), "251\nmoney\n"); + Ok(()) } #[test] -fn runtime_error_list_swap_out_of_range_is_canonical() -> Result<(), Box> { - assert_runtime_error_cli( - "def main() -> None:\n mut values = [1, 2, 3]\n values.swap(0, 99)\n", - "IndexError", - &["out of range for list"], - ) +fn runtime_error_canonicalization_cases() -> Result<(), Box> { + let cases: &[(&str, &str, &[&str])] = &[ + ( + "def main() -> None:\n let values = {\"a\": 1}\n println(values[\"b\"])\n", + "KeyError", + &["not found in dict"], + ), + ( + "def main() -> None:\n let values = [1, 2, 3]\n println(values[99])\n", + "IndexError", + &["out of range for list"], + ), + ( + "def main() -> None:\n let values = [1, 2, 3]\n println(values.index(99))\n", + "ValueError", + &["value not found in list"], + ), + ( + "def main() -> None:\n println(int(\"abc\"))\n", + "ValueError", + &["cannot convert 'abc' to int"], + ), + ( + "def main() -> None:\n println(float(\"abc\"))\n", + "ValueError", + &["cannot convert 'abc' to float"], + ), + ( + "def main() -> None:\n mut values = [1, 2, 3]\n values.remove(99)\n", + "IndexError", + &["out of range for list"], + ), + ( + "def main() -> None:\n mut values = [1, 2, 3]\n values.swap(0, 99)\n", + "IndexError", + &["out of range for list"], + ), + ]; + for (source, expected_type, expected_substrings) in cases { + assert_runtime_error_cli(source, expected_type, expected_substrings)?; + } + Ok(()) } #[test] -fn runtime_error_route_marker_runtime_misuse_is_explicit() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let web_macros_path = Path::new(env!("CARGO_MANIFEST_DIR")) - .join("crates") - .join("incan_web_macros"); - let manifest = format!( - "[project]\nname = \"route_runtime_misuse\"\nversion = \"0.3.0-dev.1\"\n\n[rust-dependencies]\nincan_web_macros = {{ path = \"{}\" }}\n", - web_macros_path.display() - ); - let src_dir = tmp.path().join("src"); - fs::create_dir_all(&src_dir)?; - fs::write(tmp.path().join("incan.toml"), manifest)?; - let main_path = src_dir.join("main.incn"); - fs::write( - &main_path, - "from std.web import route\n\ndef main() -> None:\n route(\"/users\", methods=[\"GET\"])\n", - )?; - - let check_output = Command::new(incan_debug_binary()) - .arg("--check") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - check_output.status.success(), - "expected --check to succeed so the failure is runtime.\nstderr:\n{}", - String::from_utf8_lossy(&check_output.stderr) - ); +fn assert_false_can_satisfy_typed_failure_path() -> Result<(), Box> { + let cases = [ + r#" +def fail_int(message: str) -> int: + assert false, message - let run_output = Command::new(incan_debug_binary()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - !run_output.status.success(), - "expected runtime failure, stdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&run_output.stdout), - String::from_utf8_lossy(&run_output.stderr) - ); +def main() -> None: + _ = fail_int("boom") +"#, + r#" +def fail_as[T](message: str) -> T: + assert false, message - let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&run_output.stdout)); - let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&run_output.stderr)); - let combined = format!("{stdout}\n{stderr}"); - assert!( - combined.contains("decorator marker 'incan_web_macros::route' cannot be called at runtime"), - "expected explicit decorator misuse runtime diagnostic, got:\n{combined}" - ); +def main() -> None: + _ = fail_as[int]("boom") +"#, + ]; + for source in cases { + assert_runtime_error_cli(source, "AssertionError", &["boom"])?; + } Ok(()) } @@ -2165,10 +1892,7 @@ def helper() -> Unit: panic!("failed to write test file"); }; - let Ok(output) = Command::new(incan_debug_binary()) - .args(["test", dir.to_string_lossy().as_ref()]) - .output() - else { + let Ok(output) = incan_command().args(["test", dir.to_string_lossy().as_ref()]).output() else { panic!("failed to run incan test"); }; assert!( @@ -2178,7 +1902,7 @@ def helper() -> Unit: String::from_utf8_lossy(&output.stderr) ); - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["test", "--fail-on-empty", dir.to_string_lossy().as_ref()]) .output() else { @@ -2202,7 +1926,7 @@ def main() -> None: counter += 2 println(counter) "#; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2235,7 +1959,7 @@ static counter: int = init_counter() def main() -> None: println("main") "#; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2270,7 +1994,7 @@ def main() -> None: println(len(items)) println(len(live)) "#; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2291,6 +2015,256 @@ def main() -> None: ); } +#[test] +fn test_const_model_constructor_compile_and_run_issue658() -> Result<(), Box> { + let source = r#" +model Version: + pub major: int + pub minor: int + +model Change: + pub version: Version + note [alias="message"]: FrozenStr + +model Lifecycle: + pub since: Version + pub changed: FrozenList[Change] + pub deprecated: Option[Version] + +pub const V0_1: Version = Version(major=0, minor=1) +pub const V0_3: Version = Version(major=0, minor=3) +pub const LIFECYCLE: Lifecycle = Lifecycle( + since=V0_1, + changed=[Change(version=V0_3, message="metadata")], + deprecated=None, +) + +def main() -> None: + println(f"{V0_1.major}.{V0_1.minor}") + println(f"{LIFECYCLE.changed[0].version.major}.{LIFECYCLE.changed[0].version.minor}") + println(LIFECYCLE.changed[0].note) + match LIFECYCLE.deprecated: + None => println("active") + Some(version) => println(f"{version.major}.{version.minor}") +"#; + let output = incan_command() + .args(["run", "-c", source]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + + assert!( + output.status.success(), + "expected const model constructor program to run.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!( + stdout.lines().any(|line| line.trim() == "0.1"), + "expected const model constructor output 0.1.\nstdout:\n{stdout}" + ); + assert!( + stdout.lines().any(|line| line.trim() == "0.3"), + "expected nested const model constructor output 0.3.\nstdout:\n{stdout}" + ); + assert!( + stdout.lines().any(|line| line.trim() == "metadata"), + "expected nested const model constructor output metadata.\nstdout:\n{stdout}" + ); + assert!( + stdout.lines().any(|line| line.trim() == "active"), + "expected const model option metadata output active.\nstdout:\n{stdout}" + ); + Ok(()) +} + +#[test] +fn test_lowercase_imported_pub_static_compile_and_run_issue659() -> Result<(), Box> { + let dir = make_temp_test_dir(); + let versions = dir.join("versions.incn"); + let main = dir.join("main.incn"); + std::fs::write( + &versions, + r#" +pub static v0_1: int = 1 +pub static v0_2: int = 2 +"#, + )?; + std::fs::write( + &main, + r#" +from versions import v0_1 +from versions import v0_2 as current_version + +def main() -> None: + println(v0_1) + println(current_version) +"#, + )?; + + let output = incan_command() + .args(["run", main.to_string_lossy().as_ref()]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + + assert!( + output.status.success(), + "expected lowercase imported pub static program to run.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stdout = String::from_utf8_lossy(&output.stdout); + let lines: Vec<&str> = stdout.lines().map(str::trim).filter(|line| !line.is_empty()).collect(); + assert_eq!(lines, ["1", "2"], "unexpected lowercase static output"); + Ok(()) +} + +#[test] +fn test_imported_static_initializer_does_not_deadlock_issue680() -> Result<(), Box> { + let dir = make_temp_test_dir(); + let project_name = unique_test_project_name("imported_static_deadlock"); + std::fs::write( + dir.join("incan.toml"), + format!("[project]\nname = \"{project_name}\"\nversion = \"0.1.0\"\n"), + )?; + let src_dir = dir.join("src"); + std::fs::create_dir_all(&src_dir)?; + let state = src_dir.join("state.incn"); + let facade = src_dir.join("facade.incn"); + let direct_user = src_dir.join("direct_user.incn"); + let reexport_user = src_dir.join("reexport_user.incn"); + let main = src_dir.join("main.incn"); + std::fs::write( + &state, + r#" +pub class Registry: + pub entries: list[int] + + @staticmethod + def new() -> Self: + return Registry(entries=[]) + + +pub static registry: Registry = Registry.new() + + +pub def registry_len() -> int: + return len(registry.entries) +"#, + )?; + std::fs::write(&facade, "pub from state import registry\n")?; + std::fs::write( + &direct_user, + r#" +from state import registry + + +pub def add_direct() -> None: + registry.entries.append(1) +"#, + )?; + std::fs::write( + &reexport_user, + r#" +from facade import registry + + +pub def add_reexport() -> None: + registry.entries.append(1) +"#, + )?; + std::fs::write( + &main, + r#" +from direct_user import add_direct +from reexport_user import add_reexport +from state import registry_len + + +def main() -> None: + add_direct() + add_reexport() + assert registry_len() == 2 + println("ok") +"#, + )?; + + let mut command = incan_command(); + command + .arg("run") + .arg(main.strip_prefix(&dir)?) + .current_dir(&dir) + .env("CARGO_NET_OFFLINE", "true"); + let (output, timed_out) = run_incan_command_with_timeout(command, std::time::Duration::from_secs(30))?; + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + !timed_out, + "imported static init repro timed out; likely deadlocked.\nstdout:\n{}\nstderr:\n{}", + stdout, stderr + ); + assert!( + output.status.success(), + "expected imported static init repro to run.\nstdout:\n{}\nstderr:\n{}", + stdout, + stderr + ); + assert!( + stdout.lines().any(|line| line.trim() == "ok"), + "expected imported static init repro to print ok.\nstdout:\n{stdout}" + ); + + let generated_src_dir = dir.join("target/incan").join(project_name).join("src"); + let generated_direct_user = std::fs::read_to_string(generated_src_dir.join("direct_user.rs"))?; + assert!( + generated_direct_user + .contains("use crate::state::__incan_init_module_statics as __incan_init_imported_static_registry;") + && generated_direct_user.contains("__incan_init_imported_static_registry();"), + "direct imported static access should call the defining module init guard before forcing REGISTRY:\n{}", + generated_direct_user + ); + let generated_facade = std::fs::read_to_string(generated_src_dir.join("facade.rs"))?; + assert!( + generated_facade + .contains("use crate::state::__incan_init_module_statics as __incan_init_imported_static_registry;") + && generated_facade.contains("pub(crate) fn __incan_init_module_statics()") + && generated_facade.contains("__incan_init_imported_static_registry();"), + "static re-export modules should chain the defining module init guard:\n{}", + generated_facade + ); + Ok(()) +} + +#[test] +fn test_static_list_index_assignment_and_remove_compile_and_run() -> Result<(), Box> { + let source = r#" +static entries: list[int] = [] + +def main() -> None: + entries.append(1) + entries[0] = 2 + println(entries[0]) + entries.remove(0) + entries.append(3) + println(entries[0]) +"#; + let output = incan_command() + .args(["run", "-c", source]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + + assert!( + output.status.success(), + "expected static list index mutation program to run.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stdout = String::from_utf8_lossy(&output.stdout); + let lines: Vec<&str> = stdout.lines().map(str::trim).filter(|line| !line.is_empty()).collect(); + assert_eq!(lines, ["2", "3"], "unexpected static list mutation output"); + Ok(()) +} + #[test] fn test_list_concatenation_plus_operator_runs() -> Result<(), Box> { let source = r#" @@ -2304,7 +2278,7 @@ def main() -> None: println(c[0]) println(c[3]) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -2341,7 +2315,7 @@ def main() -> None: println(find_value(True)) println(find_value(False)) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -2387,7 +2361,7 @@ def main() -> None: Some(parsed_status) => println(parsed_status.value()) None => println(0) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -2422,7 +2396,7 @@ def main() -> None: println(len(b)) println(b[0]) "#; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -2457,7 +2431,7 @@ def main() -> None: println(items[0]) println(items[1]) "#; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2496,7 +2470,7 @@ def main() -> None: println(init_order[0]) println(init_order[1]) "#; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2526,7 +2500,7 @@ mod lexer_tests { use incan_core::lang::punctuation::PunctuationId; #[test] - fn test_floor_div_tokens() { + fn lexer_token_surface_cases() { let Ok(tokens) = lex("a //= b\nc // d") else { panic!("lex failed"); }; @@ -2534,10 +2508,7 @@ mod lexer_tests { let has_floor_div = tokens.iter().any(|t| t.kind.is_operator(OperatorId::SlashSlash)); assert!(has_floor_div_eq, "expected to see //= token"); assert!(has_floor_div, "expected to see // token"); - } - #[test] - fn test_rust_style_imports() { let Ok(tokens) = lex("import foo::bar::baz as fb") else { panic!("lex failed"); }; @@ -2549,69 +2520,45 @@ mod lexer_tests { assert!(matches!(&tokens[5].kind, TokenKind::Ident(s) if s == "baz")); assert!(tokens[6].kind.is_keyword(KeywordId::As)); assert!(matches!(&tokens[7].kind, TokenKind::Ident(s) if s == "fb")); - } - #[test] - fn test_try_operator() { let Ok(tokens) = lex("result?") else { panic!("lex failed"); }; assert!(matches!(&tokens[0].kind, TokenKind::Ident(s) if s == "result")); assert!(tokens[1].kind.is_punctuation(PunctuationId::Question)); - } - #[test] - fn test_fat_arrow() { let Ok(tokens) = lex("x => y") else { panic!("lex failed"); }; assert!(tokens[1].kind.is_punctuation(PunctuationId::FatArrow)); - } - #[test] - fn test_case_keyword() { let Ok(tokens) = lex("case Some(x):") else { panic!("lex failed"); }; assert!(tokens[0].kind.is_keyword(KeywordId::Case)); - } - #[test] - fn test_pass_keyword() { let Ok(tokens) = lex("pass") else { panic!("lex failed"); }; assert!(tokens[0].kind.is_keyword(KeywordId::Pass)); - } - #[test] - fn test_mut_self() { let Ok(tokens) = lex("mut self") else { panic!("lex failed"); }; assert!(tokens[0].kind.is_keyword(KeywordId::Mut)); assert!(tokens[1].kind.is_keyword(KeywordId::SelfKw)); - } - #[test] - fn test_fstring() { let Ok(tokens) = lex(r#"f"Hello {name}""#) else { panic!("lex failed"); }; assert!(matches!(&tokens[0].kind, TokenKind::FString(_))); - } - #[test] - fn test_yield_keyword() { let Ok(tokens) = lex("yield value") else { panic!("lex failed"); }; assert!(tokens[0].kind.is_keyword(KeywordId::Yield)); assert!(matches!(&tokens[1].kind, TokenKind::Ident(s) if s == "value")); - } - #[test] - fn test_rust_keyword() { let Ok(tokens) = lex("import rust::serde_json") else { panic!("lex failed"); }; @@ -2652,7 +2599,7 @@ def main() -> None: /// End-to-end codegen tests mod codegen_tests { - use super::{incan_debug_binary, strip_ansi_escapes}; + use super::{incan_command, strip_ansi_escapes}; use incan::backend::IrCodegen; use incan::frontend::{lexer, parser, typechecker}; use std::fs; @@ -2661,7 +2608,7 @@ mod codegen_tests { use std::time::{SystemTime, UNIX_EPOCH}; fn run_incan_source(source: &str) -> std::process::Output { - Command::new(incan_debug_binary()) + incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -2740,7 +2687,7 @@ mod codegen_tests { #[test] fn test_string_literal_match_patterns_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -2788,7 +2735,7 @@ def main() -> None: #[test] fn test_payload_enum_without_equality_payload_compiles() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -2867,7 +2814,7 @@ def main() -> None: #[test] fn test_run_c_import_this() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", "import this"]) // This test should not require network access. We expect the workspace dependencies to already be available // (the test suite built them) @@ -2890,7 +2837,7 @@ def main() -> None: #[test] fn test_run_c_import_this_release_flag() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "--release", "-c", "import this"]) // This test should not require network access. We expect the workspace dependencies to already be available // (the test suite built them) @@ -2913,7 +2860,7 @@ def main() -> None: #[test] fn test_variadic_rest_calls_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -2988,6 +2935,7 @@ def main() -> None: let source = format!( r#" from std.fs import IoError, OpenOptions, Path +from std.tempfile import NamedTemporaryFile, SpooledTemporaryFile, TemporaryDirectory from rust::std::thread import sleep from rust::std::time import Duration @@ -3062,6 +3010,41 @@ def run() -> Result[None, IoError]: println(stat.modified_unix()? > 0) usage = moved.disk_usage()? println(usage.total > 0 and usage.free > 0) + + file = NamedTemporaryFile.try_new_with("incan-", ".txt", None)? + path = file.path() + path.write_text("hello", "utf-8", "strict", None)? + println(path.read_text("utf-8", "strict")?) + + directory = TemporaryDirectory.try_new_with("incan-dir-", "", None)? + child = directory.path() / "child.txt" + child.write_text("world", "utf-8", "strict", None)? + println(child.read_text("utf-8", "strict")?) + + mut memory = SpooledTemporaryFile(max_size=64) + memory.write(b"memory")? + println(memory.rolled_to_disk()) + memory.seek(0, 0)? + println(len(memory.read(-1)?)) + + mut spool = SpooledTemporaryFile(max_size=4) + spool.write(b"rolled")? + println(spool.rolled_to_disk()) + println(spool.path()?.exists()) + spool.seek(0, 0)? + println(len(spool.read(-1)?)) + kept_spool = spool.persist()? + println(kept_spool.exists()) + kept_spool.unlink()? + + kept_file = file.persist()? + println(kept_file.exists()) + kept_file.unlink()? + + kept_directory = directory.persist()? + println(kept_directory.exists()) + kept_directory.remove_tree()? + moved.remove_tree()? root.remove_tree()? return Ok(None) @@ -3075,7 +3058,7 @@ def main() -> None: copied = copied.display(), moved = moved.display() ); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source.as_str()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3111,6 +3094,16 @@ def main() -> None: "true", "true", "true", + "true", + "hello", + "world", + "false", + "6", + "true", + "true", + "6", + "true", + "true", "true" ], "unexpected std.fs output:\n{stdout}" @@ -3272,7 +3265,7 @@ def main() -> None: payload = payload.display(), missing_payload = payload.with_extension("missing").display(), ); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source.as_str()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3347,7 +3340,7 @@ def main() -> None: ::write_u32(&mut cache_anchor, 258); assert_eq!(cache_anchor, [2, 1, 0, 0]); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3442,7 +3435,7 @@ def main() -> None: #[test] fn test_std_encoding_hex_compile_and_run_strict_surface() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_encoding_hex_surface.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3476,7 +3469,7 @@ def main() -> None: #[test] fn test_std_fs_glob_string_api_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3538,7 +3531,7 @@ def main() -> None: println(cfg.retries) "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3582,7 +3575,7 @@ def main() -> None: Err(err) => println(err.message()) "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3619,7 +3612,7 @@ def main() -> None: println(BytesIO(3)) "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3660,7 +3653,7 @@ def main() -> None: println(Opener().accept("b")) "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3690,7 +3683,7 @@ def main() -> None: "#, path = path.display() ); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "-c", source.as_str()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -3709,7 +3702,7 @@ def main() -> None: #[test] fn test_match_rust_result_non_clone_payload_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3747,7 +3740,7 @@ def main() -> None: #[test] fn test_result_inspect_rust_result_non_clone_payload_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3794,7 +3787,7 @@ def main() -> None: #[test] fn test_user_authored_result_tap_borrows_callback_payload() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3848,7 +3841,7 @@ def main() -> None: #[test] fn test_std_result_helpers_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3914,7 +3907,7 @@ def main() -> None: #[test] fn test_result_methods_dogfood_std_result_helpers_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -3973,7 +3966,7 @@ def main() -> None: #[test] fn test_result_map_err_accepts_callable_object_trait_adoption() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4018,7 +4011,7 @@ def main() -> None: #[test] fn test_result_method_closure_callbacks_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4055,9 +4048,83 @@ def main() -> None: Ok(()) } + #[test] + fn test_question_mark_list_comprehension_propagates_result_issue633() -> Result<(), Box> { + let output = run_incan_source( + r#" +def parse_value(value: int) -> Result[int, str]: + if value == 2: + return Err("bad value") + return Ok(value) + + +def parse_all(values: list[int]) -> Result[list[int], str]: + return Ok([parse_value(value)? for value in values]) + + +def main() -> None: + match parse_all([1, 2, 3]): + Ok(values) => println(values[0]) + Err(err) => println(err) +"#, + ); + assert!( + output.status.success(), + "question-mark list comprehension regression failed: status={:?}\nstdout:\n{}\nstderr:\n{}", + output.status, + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); + let lines = stdout + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .collect::>(); + assert_eq!(lines, vec!["bad value"], "unexpected issue633 output:\n{stdout}"); + Ok(()) + } + + #[test] + fn test_question_mark_dict_comprehension_propagates_result_issue633() -> Result<(), Box> { + let output = run_incan_source( + r#" +def parse_key(value: int) -> Result[str, str]: + if value == 2: + return Err("bad key") + return Ok(str(value)) + + +def parse_map(values: list[int]) -> Result[dict[str, int], str]: + return Ok({parse_key(value)?: value for value in values}) + + +def main() -> None: + match parse_map([1, 2, 3]): + Ok(values) => println(values["1"]) + Err(err) => println(err) +"#, + ); + assert!( + output.status.success(), + "question-mark dict comprehension regression failed: status={:?}\nstdout:\n{}\nstderr:\n{}", + output.status, + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); + let lines = stdout + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .collect::>(); + assert_eq!(lines, vec!["bad key"], "unexpected issue633 dict output:\n{stdout}"); + Ok(()) + } + #[test] fn test_result_map_err_accepts_capturing_inline_closure() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4091,7 +4158,7 @@ def main() -> None: #[test] fn test_static_str_index_and_slice_use_string_helpers() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4121,7 +4188,7 @@ def main() -> None: #[test] fn test_collection_literal_spreads_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4157,7 +4224,7 @@ def main() -> None: #[test] fn test_enum_methods_and_trait_adoption_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4204,7 +4271,7 @@ def main() -> None: #[test] fn test_union_types_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4371,9 +4438,147 @@ def main() -> None: Ok(()) } + #[test] + fn test_union_model_variants_compile_and_run() -> Result<(), Box> { + let output = incan_command() + .args([ + "run", + "-c", + r#" +@derive(Clone) +model Leaf: + value: int + +@derive(Clone) +model Pair: + args: list[Expr] + +type Expr = Union[Leaf, Pair] + +def pair() -> Expr: + return Pair(args=[Leaf(value=1), Leaf(value=2)]) + +def clone_expr(expr: Expr) -> Expr: + return expr.clone() + +def sum_expr(expr: Expr) -> int: + match expr: + Leaf(leaf) => + return leaf.value + Pair(pair) => + return sum_expr(pair.args[0]) + +def main() -> None: + println(sum_expr(clone_expr(pair()))) +"#, + ]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + assert!( + output.status.success(), + "union model variant run-path regression failed: status={:?}\nstdout:\n{}\nstderr:\n{}", + output.status, + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + + let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); + let lines: Vec<&str> = stdout.lines().map(str::trim).filter(|line| !line.is_empty()).collect(); + assert_eq!(lines, vec!["1"], "unexpected union model variant output:\n{stdout}"); + Ok(()) + } + + #[test] + fn test_imported_union_alias_list_field_compiles_issue622() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let project_root = tmp.path().join("union_list_cross_module_alias_repro"); + fs::create_dir_all(project_root.join("src"))?; + fs::write( + project_root.join("incan.toml"), + "[project]\nname = \"union_list_cross_module_alias_repro\"\nversion = \"0.1.0\"\n", + )?; + fs::write( + project_root.join("src/exprs.incn"), + r#" +@derive(Clone) +pub model Leaf: + pub value: int + +@derive(Clone) +pub model Pair: + pub args: list[Expr] + +pub type Expr = Union[Leaf, Pair] + +pub def pair() -> Expr: + return Pair(args=[Leaf(value=1), Leaf(value=2)]) +"#, + )?; + fs::write( + project_root.join("src/lib.incn"), + r#" +from exprs import Expr, Leaf, Pair, pair + +def sum_expr(expr: Expr) -> int: + match expr: + Leaf(leaf) => return leaf.value + Pair(pair_expr) => return sum_expr(pair_expr.args[0]) + +pub def main_value() -> int: + return sum_expr(pair()) +"#, + )?; + + let output = incan_command() + .args(["build", "--lib"]) + .current_dir(&project_root) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + assert!( + output.status.success(), + "expected imported union alias list-field project to build for #622.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + Ok(()) + } + + #[test] + fn test_keyword_named_public_alias_compiles_issue669() -> Result<(), Box> { + let tmp = tempfile::tempdir()?; + let project_root = tmp.path().join("keyword_named_public_alias_repro"); + fs::create_dir_all(&project_root)?; + fs::write( + project_root.join("test_keyword_alias_probe.incn"), + r#" +pub def modulo_value(value: int) -> int: + return value + +pub mod = alias modulo_value + + +def test_keyword_alias_probe__can_call_alias() -> None: + assert mod(7) == 7, "keyword alias should call the implementation" +"#, + )?; + + let output = incan_command() + .args(["test", "test_keyword_alias_probe.incn"]) + .current_dir(&project_root) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + assert!( + output.status.success(), + "expected keyword-named public alias test project to pass for #669.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + Ok(()) + } + #[test] fn test_issue562_type_alias_dict_and_union_surfaces_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4439,7 +4644,7 @@ def main() -> None: #[test] fn test_issue502_independent_union_narrowing_branches_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4480,7 +4685,7 @@ def main() -> None: #[test] fn test_issue501_option_union_isinstance_narrowing_compile_and_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4524,7 +4729,7 @@ def main() -> None: #[test] fn test_filtered_comprehensions_run_with_borrowed_iterables() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4567,7 +4772,7 @@ def main() -> None: #[test] fn test_generator_expression_runs_lazily_with_source_ordered_clauses() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4598,7 +4803,7 @@ def main() -> None: #[test] fn test_generator_helper_chain_builds_and_runs() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4633,7 +4838,7 @@ def main() -> None: #[test] fn test_generator_function_yield_builds_and_runs() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4665,7 +4870,7 @@ def main() -> None: #[test] fn test_generator_function_body_starts_on_first_consumption() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4702,7 +4907,7 @@ def main() -> None: #[test] fn test_generic_generator_function_yield_builds_and_runs() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4732,7 +4937,7 @@ def main() -> None: #[test] fn test_clone_self_struct_field_reads_do_not_move_out_of_borrowed_self() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4765,10 +4970,62 @@ def main() -> None: Ok(()) } + #[test] + fn test_loop_item_field_index_assignment_materializes_owned_value_issue616() + -> Result<(), Box> { + let output = incan_command() + .args([ + "run", + "-c", + r#" +@derive(Clone) +model Assignment: + output_name: str + +def names(assignments: list[Assignment]) -> list[str]: + mut output_names: list[str] = [] + for assignment in assignments: + existing_idx = index_of_name(output_names, assignment.output_name) + if existing_idx >= 0: + output_names[existing_idx] = assignment.output_name + else: + output_names.append(assignment.output_name) + return output_names + +def index_of_name(names: list[str], name: str) -> int: + for idx, current in enumerate(names): + if current == name: + return idx + return -1 + +def main() -> None: + result = names([Assignment(output_name="amount"), Assignment(output_name="amount")]) + println(result[0]) +"#, + ]) + .env("CARGO_NET_OFFLINE", "true") + .output()?; + assert!( + output.status.success(), + "loop item field index-assignment regression failed: status={:?} stderr={}", + output.status, + String::from_utf8_lossy(&output.stderr) + ); + + let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); + let lines: Vec<&str> = stdout.lines().map(str::trim).filter(|line| !line.is_empty()).collect(); + assert_eq!( + lines, + vec!["amount"], + "unexpected loop item field index-assignment output:\n{stdout}" + ); + Ok(()) + } + #[test] fn test_field_backed_by_value_method_args_do_not_require_user_clone_issue241() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4809,7 +5066,7 @@ def main() -> None: #[test] fn test_issue241_generic_field_backed_method_args_infer_clone_bounds() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4851,7 +5108,7 @@ def main() -> None: #[test] fn test_returning_tuple_with_reused_field_materializes_owned_items() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4890,7 +5147,7 @@ def main() -> None: #[test] fn test_generic_tuple_return_with_reused_field_infers_clone_bound() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4929,7 +5186,7 @@ def main() -> None: #[test] fn test_incan_call_materializes_owned_value_from_box_as_ref() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -4967,7 +5224,7 @@ def main() -> None: #[test] fn test_generic_incan_call_materializes_owned_value_from_box_as_ref() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5005,7 +5262,7 @@ def main() -> None: #[test] fn test_match_on_shared_self_option_field_materializes_owned_scrutinee() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5049,7 +5306,7 @@ def main() -> None: #[test] fn test_match_on_shared_self_option_box_field_materializes_owned_scrutinee() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5094,7 +5351,7 @@ def main() -> None: #[test] fn test_generic_match_on_shared_self_option_field_infers_clone_bound() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5133,7 +5390,7 @@ def main() -> None: #[test] fn test_trait_supertraits_runtime_with_backend_clone_bounds() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5181,7 +5438,7 @@ def main() -> None: #[test] fn test_result_ok_string_literals_run_without_manual_str_wrapping() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -5234,7 +5491,7 @@ def main() -> None: "#, )?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "--release", source_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -5254,10 +5511,9 @@ def main() -> None: } #[test] - fn test_build_web_route_uses_proc_macro_passthrough() { + fn test_check_web_route_uses_proc_macro_passthrough() { let project_dir = make_temp_dir("incan_web_proc_macro_test"); let source_path = project_dir.join("main.incn"); - let out_dir = project_dir.join("out"); let source = r#" import std.async from std.web import route @@ -5273,44 +5529,20 @@ def main() -> None: panic!("failed to write source file"); }; - let Ok(output) = Command::new(incan_debug_binary()) - .args([ - "build", - source_path.to_string_lossy().as_ref(), - out_dir.to_string_lossy().as_ref(), - ]) + let Ok(output) = incan_command() + .args(["--check", source_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output() else { - panic!("failed to run incan build"); + panic!("failed to run incan check"); }; assert!( output.status.success(), - "incan build web route failed: status={:?} stderr={}", + "incan check web route failed: status={:?} stderr={}", output.status, String::from_utf8_lossy(&output.stderr) ); - - let generated_main = out_dir.join("src/main.rs"); - let Ok(main_rs) = std::fs::read_to_string(&generated_main) else { - panic!("failed to read generated Rust source"); - }; - assert!( - main_rs.contains("#[incan_web_macros::route("), - "expected generated web route to use proc macro passthrough:\n{}", - main_rs - ); - assert!( - !main_rs.contains("__incan_router!"), - "legacy __incan_router! macro should not be emitted:\n{}", - main_rs - ); - assert!( - !main_rs.contains("set_router"), - "legacy set_router() call should not be emitted:\n{}", - main_rs - ); } #[test] @@ -5378,7 +5610,7 @@ async def main() -> None: "#; std::fs::write(&source_path, source)?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", source_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -5465,7 +5697,7 @@ async def main() -> Result[None, str]: panic!("failed to write source file"); }; - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args([ "build", source_path.to_string_lossy().as_ref(), @@ -5538,7 +5770,7 @@ def main() -> None: )?; let out_dir = project_dir.join("out"); - let build_output = Command::new(incan_debug_binary()) + let build_output = incan_command() .args([ "build", main_path.to_string_lossy().as_ref(), @@ -5575,7 +5807,7 @@ def main() -> None: "expected nested keyword module path attr in api/mod.rs, got:\n{api_mod_rs}" ); - let run_output = Command::new(incan_debug_binary()) + let run_output = incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -5665,7 +5897,7 @@ async def main() -> None: "#; std::fs::write(&source_path, source)?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", source_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -5793,7 +6025,7 @@ async def main() -> None: "#; std::fs::write(&source_path, source)?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", source_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -5832,7 +6064,7 @@ async def main() -> None: #[test] fn test_run_repro_model_traits() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/repro_model_traits.incn"]) // This should not require network access (workspace deps should already be available). .env("CARGO_NET_OFFLINE", "true") @@ -5859,7 +6091,7 @@ async def main() -> None: /// RFC 021: Runtime verification that __fields__() returns correct FieldInfo values #[test] fn test_run_field_info_reflection() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/field_info_reflection.incn"]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -5941,7 +6173,7 @@ async def main() -> None: /// RFC 023: Runtime parity check for source-defined stdlib surfaces migrated off helper stubs. #[test] fn test_run_rfc023_stdlib_behavior_parity() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/rfc023_stdlib_behavior_parity.incn"]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -5991,7 +6223,7 @@ async def main() -> None: #[test] fn test_run_rfc030_std_collections_behavior() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/rfc030_std_collections_behavior.incn"]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -6009,7 +6241,7 @@ async def main() -> None: #[test] fn test_run_rfc064_std_encoding_behavior() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/rfc064_std_encoding_behavior.incn"]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -6035,7 +6267,7 @@ async def main() -> None: #[test] fn test_run_std_uuid_surface() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_uuid_surface.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -6052,7 +6284,7 @@ async def main() -> None: #[test] fn test_run_std_ordinal_map_surface() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_ordinal_map_surface.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -6082,9 +6314,8 @@ async def main() -> None: #[test] fn test_run_std_regex_rfc059_surface() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_regex_surface.incn"]) - .env("CARGO_NET_OFFLINE", "true") .output()?; assert!( @@ -6123,12 +6354,37 @@ async def main() -> None: ], "unexpected std.regex output:\n{stdout}" ); + let generated_core = fs::read_to_string("target/incan/std_regex_surface/src/__incan_std/regex/_core.rs")?; + for unexpected in [ + "RegexBuilder::new(&(pattern).to_string())", + "raw.find(&(text).to_string())", + "raw.find_iter(&(text).to_string())", + "raw.captures(&(text).to_string())", + "raw.captures_iter(&(text).to_string())", + ] { + assert!( + !generated_core.contains(unexpected), + "std.regex should let the compiler borrow Incan strings for Rust regex APIs instead of cloning them:\n{generated_core}" + ); + } + for expected in [ + "RegexBuilder::new(&pattern)", + "raw.find(&text)", + "raw.find_iter(&text)", + "raw.captures(&text)", + "raw.captures_iter(&text)", + ] { + assert!( + generated_core.contains(expected), + "std.regex should preserve compiler-managed Rust borrow boundaries; missing `{expected}`:\n{generated_core}" + ); + } Ok(()) } #[test] fn test_run_std_regex_unsupported_safe_engine_pattern_reports_error() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "run", "-c", @@ -6144,7 +6400,6 @@ def main() -> None: println(err.message()) "#, ]) - .env("CARGO_NET_OFFLINE", "true") .output()?; assert!( @@ -6172,7 +6427,7 @@ def main() -> None: #[test] fn test_run_u128_modulo_floor_div() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/u128_modulo_floor_div.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -6189,7 +6444,7 @@ def main() -> None: #[test] fn test_run_rfc030_field_overlay_reflection() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args(["run", "tests/fixtures/rfc030_field_overlay_reflection.incn"]) .env("CARGO_NET_OFFLINE", "true") .output() @@ -6210,7 +6465,7 @@ def main() -> None: let project_dir = make_temp_dir("incan_cycle_explicit_call_site_check"); let main_path = super::write_cycle_explicit_call_site_generics_project(&project_dir)?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .arg("--check") .arg(main_path) .env("CARGO_NET_OFFLINE", "true") @@ -6230,7 +6485,7 @@ def main() -> None: let project_dir = make_temp_dir("incan_cycle_explicit_call_site_run"); let main_path = super::write_cycle_explicit_call_site_generics_project(&project_dir)?; - let output = Command::new(incan_debug_binary()) + let output = incan_command() .arg("run") .arg(main_path) .env("CARGO_NET_OFFLINE", "true") @@ -6309,7 +6564,7 @@ def main() -> None: #[test] fn test_const_declarations_compile_and_run() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args([ "run", "-c", @@ -6360,7 +6615,7 @@ def main() -> None: #[test] fn test_const_str_materializes_to_owned_str_at_runtime_sites() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args([ "run", "-c", @@ -6553,7 +6808,7 @@ def main() -> None: #[test] fn test_mixed_numeric_codegen_runs() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args([ "run", "-c", @@ -6573,96 +6828,30 @@ def main() -> None: assert!( output.status.success(), - "mixed numeric run failed: status={:?} stderr={}", - output.status, - String::from_utf8_lossy(&output.stderr) - ); - - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains('3'), - "mixed numeric output missing expected result; stdout={}", - stdout - ); - } - - #[test] - fn test_std_async_race_helper_first_completion_runs() { - let output = run_incan_source( - r#" -from std.async.race import arm, race -from std.async.time import sleep - -def label(value: int) -> str: - return f"win:{value}" - -async def fast() -> int: - return 1 - -async def slow() -> int: - await sleep(0.01) - return 2 - -async def main() -> None: - println(await race(arm(slow(), label), arm(fast(), label))) -"#, - ); - assert!( - output.status.success(), - "std.async.race first-completion run failed: status={:?}\nstdout:\n{}\nstderr:\n{}", - output.status, - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); - assert_eq!( - stdout.lines().last().map(str::trim), - Some("win:1"), - "unexpected stdout:\n{stdout}" - ); - } - - #[test] - fn test_std_async_race_helper_ready_tie_uses_source_order() { - let output = run_incan_source( - r#" -from std.async.race import arm, race - -def label(value: int) -> str: - return f"win:{value}" - -async def first() -> int: - return 1 - -async def second() -> int: - return 2 - -async def main() -> None: - println(await race(arm(first(), label), arm(second(), label))) -"#, - ); - assert!( - output.status.success(), - "std.async.race ready-tie run failed: status={:?}\nstdout:\n{}\nstderr:\n{}", + "mixed numeric run failed: status={:?} stderr={}", output.status, - String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); - let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); - assert_eq!( - stdout.lines().last().map(str::trim), - Some("win:1"), - "unexpected stdout:\n{stdout}" + + let stdout = String::from_utf8_lossy(&output.stdout); + assert!( + stdout.contains('3'), + "mixed numeric output missing expected result; stdout={}", + stdout ); } #[test] - fn test_race_for_expression_first_completion_runs_through_shared_runtime() { + fn test_std_async_race_and_race_for_surfaces_share_one_run() { let output = run_incan_source( r#" import std.async +from std.async.race import arm, race from std.async.time import sleep +def label(value: int) -> str: + return f"win:{value}" + async def fast() -> int: return 1 @@ -6670,66 +6859,48 @@ async def slow() -> int: await sleep(0.01) return 2 -async def main() -> None: - prefix = "win" - result = race for value: - await slow() => f"{prefix}:{value}" - await fast() => f"{prefix}:{value}" - println(result) -"#, - ); - assert!( - output.status.success(), - "race for first-completion run failed: status={:?}\nstdout:\n{}\nstderr:\n{}", - output.status, - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); - assert_eq!( - stdout.lines().last().map(str::trim), - Some("win:1"), - "unexpected stdout:\n{stdout}" - ); - } - - #[test] - fn test_race_for_expression_ready_tie_uses_stdlib_source_order() { - let output = run_incan_source( - r#" -import std.async - async def first() -> int: return 1 async def second() -> int: return 2 -async def main() -> None: - result = race for value: +async def run_race_for_first() -> str: + prefix = "win" + return race for value: + await slow() => f"{prefix}:{value}" + await fast() => f"{prefix}:{value}" + +async def run_race_for_tie() -> int: + return race for value: await first() => value await second() => value - println(result) + +async def main() -> None: + println(await race(arm(slow(), label), arm(fast(), label))) + println(await race(arm(first(), label), arm(second(), label))) + println(await run_race_for_first()) + println(await run_race_for_tie()) "#, ); assert!( output.status.success(), - "race for ready-tie run failed: status={:?}\nstdout:\n{}\nstderr:\n{}", + "std.async race surface batch failed: status={:?}\nstdout:\n{}\nstderr:\n{}", output.status, String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); let stdout = strip_ansi_escapes(&String::from_utf8_lossy(&output.stdout)); assert_eq!( - stdout.lines().last().map(str::trim), - Some("1"), + stdout.lines().map(str::trim).collect::>(), + vec!["win:1", "win:1", "win:1", "1"], "unexpected stdout:\n{stdout}" ); } #[test] - fn test_std_math_module_constants_and_functions_run() { - let Ok(output) = Command::new(incan_debug_binary()) + fn test_std_math_surface_runs() { + let Ok(output) = incan_command() .args([ "run", "-c", @@ -6744,6 +6915,19 @@ def main() -> None: println(math.hypot(3.0, 4.0)) println(math.gcd(54, 24)) println(math.lcm(6, 8)) + + assert math.is_int_like("0") + assert math.is_int_like("-123") + assert not math.is_int_like("1e3") + assert not math.is_int_like("01") + + assert math.is_float_like("0.0") + assert math.is_float_like("-0.5") + assert math.is_float_like("1e3") + assert math.is_float_like("1.25E+10") + assert not math.is_float_like("1") + assert not math.is_float_like("+1") + assert not math.is_float_like("1e+") "#, ]) .env("CARGO_NET_OFFLINE", "true") @@ -6801,43 +6985,6 @@ def main() -> None: assert_eq!(lcm, 24, "unexpected lcm value: {lcm}"); } - #[test] - fn test_std_math_numeric_like_helpers_run() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) - .args([ - "run", - "-c", - r#" -import std.math - -def main() -> None: - assert math.is_int_like("0") - assert math.is_int_like("-123") - assert not math.is_int_like("1e3") - assert not math.is_int_like("01") - - assert math.is_float_like("0.0") - assert math.is_float_like("-0.5") - assert math.is_float_like("1e3") - assert math.is_float_like("1.25E+10") - assert not math.is_float_like("1") - assert not math.is_float_like("+1") - assert not math.is_float_like("1e+") -"#, - ]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "std.math numeric-like helper run failed: status={:?}\nstdout={}\nstderr={}", - output.status, - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - Ok(()) - } - #[test] fn test_std_datetime_surface_runs_with_std_time_runtime_boundary() -> Result<(), Box> { let runtime_source = std::fs::read_to_string("crates/incan_stdlib/stdlib/datetime/runtime.incn")?; @@ -6861,7 +7008,7 @@ def main() -> None: "std.datetime civil calendar code must remain source-defined Incan" ); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_datetime_surface.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -6945,7 +7092,7 @@ def main() -> None: let mut snappy = snap::raw::Encoder::new(); assert!(!snappy.compress_vec(sample)?.is_empty()); - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args(["run", "tests/fixtures/valid/std_compression_surface.incn"]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -6982,7 +7129,7 @@ def main() -> None: #[test] fn test_rust_associated_call_in_elif_branch_uses_path_syntax() { - let Ok(output) = Command::new(incan_debug_binary()) + let Ok(output) = incan_command() .args([ "run", "-c", @@ -7023,9 +7170,8 @@ def main() -> None: /// stdout/stderr/exit code. They catch integration bugs like broken per-file `cargo test` harness wiring or parametrize /// expansion that unit tests cannot detect. mod test_runner_e2e { - use super::incan_debug_binary; + use super::incan_command; use std::path::Path; - use std::process::Command; use std::sync::atomic::{AtomicU64, Ordering}; static TEST_PROJECT_COUNTER: AtomicU64 = AtomicU64::new(0); @@ -7057,13 +7203,20 @@ mod test_runner_e2e { /// Run `incan test` for the given path argument (file or directory). fn run_incan_test_path(path: &Path) -> std::process::Output { - Command::new(incan_debug_binary()) + incan_command() .args(["test", path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") + .env("INCAN_TEST_SHARED_TARGET_DIR", shared_test_runner_target_dir()) .output() .unwrap_or_else(|e| panic!("failed to run `incan test`: {}", e)) } + fn shared_test_runner_target_dir() -> std::path::PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("incan_e2e_shared_target") + } + /// Run `incan test` on a directory and return the combined output. fn run_incan_test(dir: &Path) -> std::process::Output { run_incan_test_path(dir) @@ -7071,23 +7224,25 @@ mod test_runner_e2e { /// Run `incan test` with extra flags. fn run_incan_test_with_args(dir: &Path, extra: &[&str]) -> std::process::Output { - let mut cmd = Command::new(incan_debug_binary()); + let mut cmd = incan_command(); cmd.arg("test"); for arg in extra { cmd.arg(arg); } cmd.arg(dir.to_string_lossy().as_ref()); cmd.env("CARGO_NET_OFFLINE", "true"); + cmd.env("INCAN_TEST_SHARED_TARGET_DIR", shared_test_runner_target_dir()); cmd.output() .unwrap_or_else(|e| panic!("failed to run `incan test`: {}", e)) } /// Run `incan test` with `cwd` and a relative path argument. fn run_incan_test_relative(cwd: &Path, relative_path: &str) -> std::process::Output { - Command::new(incan_debug_binary()) + incan_command() .arg("test") .arg(relative_path) .env("CARGO_NET_OFFLINE", "true") + .env("INCAN_TEST_SHARED_TARGET_DIR", shared_test_runner_target_dir()) .current_dir(cwd) .output() .unwrap_or_else(|e| panic!("failed to run `incan test {relative_path}`: {}", e)) @@ -7095,7 +7250,7 @@ mod test_runner_e2e { /// Run `incan build ` for an inline-test production source. fn run_incan_build(entry: &Path, out_dir: &Path) -> std::process::Output { - let output = Command::new(incan_debug_binary()) + let output = incan_command() .args([ "build", entry.to_string_lossy().as_ref(), @@ -7112,46 +7267,33 @@ mod test_runner_e2e { // ---- Passing test ---- #[test] - fn e2e_passing_test_succeeds() { + fn e2e_basic_reporting_decorator_filter_and_capture_share_one_project() { let dir = write_test_project( - "test_math.incn", + "test_runner_surface.incn", r#" -from std.testing import assert_eq +from std.testing import assert_eq, test def test_addition() -> None: assert_eq(1 + 1, 2) -"#, - ); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); +def test_one() -> None: + assert_eq(1, 1) - assert!( - output.status.success(), - "expected passing test to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains("PASSED") || stdout.contains("passed"), - "expected PASSED in output.\nstdout:\n{}", - stdout, - ); - } +def test_two() -> None: + assert_eq(2, 2) - #[test] - fn e2e_two_tests_in_one_file_share_single_cargo_batch() { - let dir = write_test_project( - "test_pair.incn", - r#" -from std.testing import assert_eq +@test +def verifies_total() -> None: + assert_eq(40 + 2, 42) -def test_one() -> None: +def test_alpha() -> None: assert_eq(1, 1) -def test_two() -> None: +def test_beta() -> None: assert_eq(2, 2) + +def test_prints() -> None: + print("VISIBLE_CAPTURE") "#, ); @@ -7166,15 +7308,60 @@ def test_two() -> None: stderr, ); assert!( - stdout.contains("test_pair.incn::test_one") && stdout.contains("test_pair.incn::test_two"), - "expected each test name in reporter output.\nstdout:\n{}", + stdout.contains("PASSED") || stdout.contains("passed"), + "expected PASSED in output.\nstdout:\n{}", + stdout, + ); + assert!( + stdout.contains("test_runner_surface.incn::test_one") + && stdout.contains("test_runner_surface.incn::test_two") + && stdout.contains("test_runner_surface.incn::verifies_total"), + "expected basic and decorated test names in reporter output.\nstdout:\n{}", stdout, ); assert!( - stdout.match_indices("PASSED").count() >= 2, - "expected two passing results (per-test PASSED lines).\nstdout:\n{}", + stdout.match_indices("PASSED").count() >= 6, + "expected passing result lines for all basic surface tests.\nstdout:\n{}", stdout, ); + + let listed = run_incan_test_with_args(&dir, &["--list", "-k", "test_beta"]); + let listed_stdout = String::from_utf8_lossy(&listed.stdout); + let listed_stderr = String::from_utf8_lossy(&listed.stderr); + assert!( + listed.status.success(), + "expected --list -k run to succeed.\nstdout:\n{}\nstderr:\n{}", + listed_stdout, + listed_stderr, + ); + assert!( + listed_stdout + .lines() + .any(|line| line == "test_runner_surface.incn::test_beta"), + "expected exact listed beta id rooted at the explicit test directory.\nstdout:\n{}", + listed_stdout, + ); + assert!( + !listed_stdout.contains(dir.to_string_lossy().as_ref()), + "expected --list output to avoid machine-local absolute paths.\nstdout:\n{}", + listed_stdout, + ); + assert!( + !listed_stdout.contains("test_runner_surface.incn::test_alpha"), + "expected keyword filter to hide alpha.\nstdout:\n{}", + listed_stdout, + ); + + let captured = run_incan_test_with_args(&dir, &["--nocapture", "-k", "test_prints"]); + let captured_stdout = String::from_utf8_lossy(&captured.stdout); + let captured_stderr = String::from_utf8_lossy(&captured.stderr); + assert!( + captured.status.success(), + "expected nocapture run to succeed.\nstdout:\n{}\nstderr:\n{}", + captured_stdout, + captured_stderr, + ); + assert!(captured_stdout.contains("VISIBLE_CAPTURE")); } #[test] @@ -7267,6 +7454,57 @@ def test_b() -> None: Ok(()) } + #[test] + fn e2e_cross_file_batch_rebases_spans_for_type_info_issue692() -> Result<(), Box> { + fn source_with_call_offset(header: &str, call_prefix: &str, call_and_tail: &str, offset: usize) -> String { + let fixed_len = header.len() + call_prefix.len(); + assert!( + offset >= fixed_len + 6, + "test fixture offset leaves no room for padding" + ); + let padding = format!(" #{}\n", "x".repeat(offset - fixed_len - 6)); + format!("{header}{padding}{call_prefix}{call_and_tail}") + } + + let target_offset = 320; + let dir = write_test_project( + "test_constructor_marker.incn", + &source_with_call_offset( + "model Box:\n value: int\n\ndef test_type_constructor() -> None:\n", + " item = ", + "Box(value=1)\n assert item.value == 1\n", + target_offset, + ), + ); + std::fs::write( + dir.join("test_zero_arg_call.incn"), + source_with_call_offset( + "def tap() -> str:\n return \"ok\"\n\ndef test_zero_arg_call_in_list() -> None:\n", + " values = [", + "tap()]\n assert values[0] == \"ok\"\n", + target_offset, + ), + )?; + + let output = run_incan_test(&dir); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + + assert!( + output.status.success(), + "expected same-span constructor and zero-argument calls from different files not to share type-info facts.\nstdout:\n{}\nstderr:\n{}", + stdout, + stderr, + ); + assert!( + stdout.contains("test_constructor_marker.incn::test_type_constructor") + && stdout.contains("test_zero_arg_call.incn::test_zero_arg_call_in_list"), + "expected both files to run in one directory test batch.\nstdout:\n{}", + stdout, + ); + Ok(()) + } + #[test] fn e2e_imported_default_expression_expands_with_required_scope_issue395() -> Result<(), Box> { @@ -7303,129 +7541,56 @@ pub def combine(left: int, middle: int = fallback(), right: int = 3) -> int: from std.testing import assert_eq from helper import combine -def test_imported_default_expression_expands_with_required_imports() -> None: - assert_eq(combine(left=1, right=4), 7, "default expression helper should be available after expansion") -"#, - )?; - - let output = run_incan_test_relative(&dir, "tests"); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - - assert!( - output.status.success(), - "expected imported default expression test to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains( - "test_default_expr_import.incn::test_imported_default_expression_expands_with_required_imports" - ), - "expected issue 395 test name in reporter output.\nstdout:\n{}", - stdout, - ); - Ok(()) - } - - #[test] - fn e2e_explicit_test_decorator_discovers_non_prefixed_function() { - let dir = write_test_project( - "test_decorator.incn", - r#" -from std.testing import assert_eq, test - -@test -def verifies_total() -> None: - assert_eq(40 + 2, 42) -"#, - ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - - assert!( - output.status.success(), - "expected @test-decorated function to run.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains("test_decorator.incn::verifies_total"), - "expected decorated test id in output.\nstdout:\n{}", - stdout, - ); - } - - #[test] - fn e2e_list_and_keyword_filter_use_stable_test_ids() { - let dir = write_test_project( - "test_list_filter.incn", - r#" -from std.testing import assert_eq - -def test_alpha() -> None: - assert_eq(1, 1) - -def test_beta() -> None: - assert_eq(2, 2) +def test_imported_default_expression_expands_with_required_imports() -> None: + assert_eq(combine(left=1, right=4), 7, "default expression helper should be available after expansion") "#, - ); + )?; - let output = run_incan_test_with_args(&dir, &["--list", "-k", "test_beta"]); + let output = run_incan_test_relative(&dir, "tests"); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected --list -k run to succeed.\nstdout:\n{}\nstderr:\n{}", + "expected imported default expression test to succeed.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); assert!( - stdout.lines().any(|line| line == "test_list_filter.incn::test_beta"), - "expected exact listed beta id rooted at the explicit test directory.\nstdout:\n{}", - stdout, - ); - assert!( - !stdout.contains(dir.to_string_lossy().as_ref()), - "expected --list output to avoid machine-local absolute paths.\nstdout:\n{}", - stdout, - ); - assert!( - !stdout.contains("test_list_filter.incn::test_alpha"), - "expected keyword filter to hide alpha.\nstdout:\n{}", + stdout.contains( + "test_default_expr_import.incn::test_imported_default_expression_expands_with_required_imports" + ), + "expected issue 395 test name in reporter output.\nstdout:\n{}", stdout, ); + Ok(()) } #[test] - fn e2e_json_format_emits_result_records() -> Result<(), Box> { + fn e2e_report_formats_share_one_project() -> Result<(), Box> { let dir = write_test_project( - "test_json_report.incn", + "test_report_formats.incn", r#" from std.testing import assert_eq -def test_json_one() -> None: +def test_report_one() -> None: assert_eq(1, 1) "#, ); - let output = run_incan_test_with_args(&dir, &["--format", "json", "--shuffle", "--seed", "7"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - + let json_output = run_incan_test_with_args(&dir, &["--format", "json", "--shuffle", "--seed", "7"]); + let json_stdout = String::from_utf8_lossy(&json_output.stdout); + let json_stderr = String::from_utf8_lossy(&json_output.stderr); assert!( - output.status.success(), + json_output.status.success(), "expected JSON-format run to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + json_stdout, + json_stderr, ); let mut saw_result = false; let mut saw_summary = false; - for line in stdout.lines().filter(|line| !line.trim().is_empty()) { + for line in json_stdout.lines().filter(|line| !line.trim().is_empty()) { let value: serde_json::Value = serde_json::from_str(line)?; if value.get("test_id").is_some() { saw_result = true; @@ -7435,7 +7600,7 @@ def test_json_one() -> None: ); assert_eq!( value.get("test_id").and_then(|v| v.as_str()), - Some("test_json_report.incn::test_json_one") + Some("test_report_formats.incn::test_report_one") ); assert_eq!(value.get("status").and_then(|v| v.as_str()), Some("passed")); } @@ -7450,47 +7615,31 @@ def test_json_one() -> None: ); } } - assert!( saw_result, "expected at least one JSON result record.\nstdout:\n{}", - stdout + json_stdout ); - assert!(saw_summary, "expected a JSON summary record.\nstdout:\n{}", stdout); - Ok(()) - } - - #[test] - fn e2e_junit_report_writes_testcase_xml() { - let dir = write_test_project( - "test_junit_report.incn", - r#" -from std.testing import assert_eq + assert!(saw_summary, "expected a JSON summary record.\nstdout:\n{}", json_stdout); -def test_junit_one() -> None: - assert_eq(1, 1) -"#, - ); let report = dir.join("reports").join("junit.xml"); let report_arg = report.to_string_lossy().to_string(); - let output = run_incan_test_with_args(&dir, &["--junit", report_arg.as_str()]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - + let junit_output = run_incan_test_with_args(&dir, &["--junit", report_arg.as_str()]); + let junit_stdout = String::from_utf8_lossy(&junit_output.stdout); + let junit_stderr = String::from_utf8_lossy(&junit_output.stderr); assert!( - output.status.success(), + junit_output.status.success(), "expected JUnit report run to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + junit_stdout, + junit_stderr, ); - let Ok(xml) = std::fs::read_to_string(&report) else { - panic!("failed to read {}", report.display()); - }; + let xml = std::fs::read_to_string(&report)?; assert!( - xml.contains(" None: } #[test] - fn e2e_conftest_fixture_is_visible_to_nested_tests() { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "conftest_fixture" -version = "0.1.0" -"#, - ); - let tests_dir = dir.join("tests").join("unit"); - if let Err(err) = std::fs::create_dir_all(&tests_dir) { - panic!("failed to create tests dir: {}", err); - } - if let Err(err) = std::fs::write( - dir.join("tests").join("conftest.incn"), - r#" -from std.testing import fixture - -@fixture -def answer() -> int: - return 42 -"#, - ) { - panic!("failed to write conftest: {}", err); - } - if let Err(err) = std::fs::write( - tests_dir.join("test_answer.incn"), - r#" -from std.testing import assert_eq - -def test_answer(answer: int) -> None: - assert_eq(answer, 42) -"#, - ) { - panic!("failed to write nested test: {}", err); - } - - let output = run_incan_test_relative(&dir, "tests"); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected conftest fixture injection to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains("test_answer.incn::test_answer"), - "expected nested stable id in output.\nstdout:\n{}", - stdout, - ); - } - - #[test] - fn e2e_nested_test_root_uses_same_conftest_boundary_for_collection_and_execution() { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "nested_conftest_boundary" -version = "0.1.0" -"#, - ); - let tests_dir = dir.join("tests"); - let unit_dir = tests_dir.join("unit"); - if let Err(err) = std::fs::create_dir_all(&unit_dir) { - panic!("failed to create nested tests dir: {}", err); - } - if let Err(err) = std::fs::write( - tests_dir.join("conftest.incn"), - r#" -from std.testing import fixture - -@fixture -def answer() -> int: - return 1 -"#, - ) { - panic!("failed to write parent conftest: {}", err); - } - if let Err(err) = std::fs::write( - unit_dir.join("conftest.incn"), - r#" -from std.testing import fixture - -@fixture -def answer() -> int: - return 2 -"#, - ) { - panic!("failed to write nested conftest: {}", err); - } - if let Err(err) = std::fs::write( - unit_dir.join("test_value.incn"), - r#" -from std.testing import assert_eq - -def test_answer(answer: int) -> None: - assert_eq(answer, 2) -"#, - ) { - panic!("failed to write nested conftest test: {}", err); - } - - let output = run_incan_test(&unit_dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected nested root run to use only root-bounded conftest sources.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - } - - #[test] - fn e2e_nested_conftest_fixture_overrides_parent_fixture() { - let dir = write_test_project( + fn e2e_conftest_nearest_fixture_override_project() { + let override_dir = write_test_project( "incan.toml", r#"[project] name = "nested_conftest_precedence" version = "0.1.0" "#, ); - let tests_dir = dir.join("tests"); - let unit_dir = tests_dir.join("unit"); - if let Err(err) = std::fs::create_dir_all(&unit_dir) { + let override_tests_dir = override_dir.join("tests"); + let override_unit_dir = override_tests_dir.join("unit"); + if let Err(err) = std::fs::create_dir_all(&override_unit_dir) { panic!("failed to create nested tests dir: {}", err); } if let Err(err) = std::fs::write( - tests_dir.join("conftest.incn"), + override_tests_dir.join("conftest.incn"), r#" from std.testing import fixture @@ -7673,7 +7708,7 @@ def shared() -> str: panic!("failed to write parent conftest: {}", err); } if let Err(err) = std::fs::write( - unit_dir.join("conftest.incn"), + override_unit_dir.join("conftest.incn"), r#" from std.testing import fixture @@ -7685,7 +7720,7 @@ def shared() -> str: panic!("failed to write nested conftest: {}", err); } if let Err(err) = std::fs::write( - unit_dir.join("test_precedence.incn"), + override_unit_dir.join("test_precedence.incn"), r#" from std.testing import assert_eq @@ -7696,7 +7731,7 @@ def test_uses_nearest_fixture(shared: str) -> None: panic!("failed to write nested conftest test: {}", err); } - let output = run_incan_test(&dir); + let output = run_incan_test(&override_dir); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( @@ -7709,35 +7744,16 @@ def test_uses_nearest_fixture(shared: str) -> None: } #[test] - fn e2e_builtin_tmp_path_fixture_is_injected() { + fn e2e_builtin_fixture_and_assert_helper_share_one_project() { let dir = write_test_project( - "test_tmp_path.incn", + "test_builtin_fixture_and_assert_helper.incn", r#" from std.testing import assert_eq +import std.testing as testing from rust::std::path import PathBuf def test_tmp_path_fixture(tmp_path: PathBuf) -> None: assert_eq(tmp_path.exists(), true) -"#, - ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected built-in tmp_path fixture to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - } - - #[test] - fn e2e_std_testing_assert_helper_is_normalized_before_codegen() { - let dir = write_test_project( - "test_assert_helper.incn", - r#" -import std.testing as testing def test_assert_helper() -> None: testing.assert(True) @@ -7749,82 +7765,30 @@ def test_assert_helper() -> None: let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected one-argument std.testing.assert call to run without generated Rust string rewriting.\nstdout:\n{}\nstderr:\n{}", + "expected built-in tmp_path fixture to succeed.\nstdout:\n{}\nstderr:\n{}", stdout, - stderr + stderr, ); assert!(stdout.contains("test_assert_helper")); } #[test] - fn e2e_marker_expr_and_strict_markers_use_conftest_registry() { + fn e2e_markers_parametrize_timeout_and_collection_errors_share_projects() { + let platform = std::env::consts::OS; let dir = write_test_project( - "incan.toml", - r#"[project] -name = "strict_markers" -version = "0.1.0" -"#, - ); - let tests_dir = dir.join("tests"); - if let Err(err) = std::fs::create_dir_all(&tests_dir) { - panic!("failed to create tests dir: {}", err); - } - if let Err(err) = std::fs::write( - tests_dir.join("conftest.incn"), - r#" -const TEST_MARKERS: List[str] = ["smoke"] + "test_runner_collection_surface.incn", + &format!( + r#" +from rust::std::thread import sleep +from rust::std::time import Duration +from std.testing import assert_eq, feature, mark, param_case, parametrize, platform, skipif, slow, timeout, xfail, xfailif + +const TEST_MARKERS: List[str] = ["api", "db", "smoke"] const TEST_MARKS: List[str] = ["smoke"] -"#, - ) { - panic!("failed to write conftest: {}", err); - } - if let Err(err) = std::fs::write( - tests_dir.join("test_markers.incn"), - r#" -from std.testing import assert_eq def test_inherited_smoke() -> None: assert_eq(1, 1) -def test_other() -> None: - assert_eq(1, 1) -"#, - ) { - panic!("failed to write marker test: {}", err); - } - - let listed = run_incan_test_with_args(&tests_dir, &["--list", "-m", "smoke", "--strict-markers"]); - let stdout = String::from_utf8_lossy(&listed.stdout); - let stderr = String::from_utf8_lossy(&listed.stderr); - assert!( - listed.status.success(), - "expected strict registered marker list to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!(stdout.contains("test_markers.incn::test_inherited_smoke")); - - let strict_error = run_incan_test_with_args(&tests_dir, &["--list", "-m", "missing", "--strict-markers"]); - let strict_stdout = String::from_utf8_lossy(&strict_error.stdout); - let strict_stderr = String::from_utf8_lossy(&strict_error.stderr); - assert!( - !strict_error.status.success(), - "expected unknown strict marker to fail.\nstdout:\n{}\nstderr:\n{}", - strict_stdout, - strict_stderr, - ); - assert!(strict_stderr.contains("unknown marker `missing`")); - } - - #[test] - fn e2e_marker_expr_boolean_grammar_filters_tests() -> Result<(), Box> { - let dir = write_test_project( - "test_marker_expr.incn", - r#" -from std.testing import assert_eq, mark, slow - -const TEST_MARKERS: List[str] = ["api", "db"] - @mark("api") def test_api() -> None: assert_eq(1, 1) @@ -7837,51 +7801,84 @@ def test_api_slow() -> None: @mark("db") def test_db() -> None: assert_eq(1, 1) -"#, + +def test_fast() -> None: + assert_eq(1, 1) + +@slow +def test_slow_case() -> None: + assert_eq(1, 1) + +@parametrize("x, expected", [ + param_case((1, 3), marks=[xfail("known")], id="one-three"), + (2, 4), +], ids=["ignored", "two-four"]) +def test_marked_double(x: int, expected: int) -> None: + assert_eq(x * 2, expected) + +@parametrize("x", [1, 2], ids=["one", "two"]) +@parametrize("y", [10, 20], ids=["ten", "twenty"]) +def test_pair(x: int, y: int) -> None: + assert_eq(x < y, true) + +@parametrize("a, b, expected", [(1, 2, 3), (10, 20, 30), (0, 0, 0)]) +def test_add(a: int, b: int, expected: int) -> None: + assert_eq(a + b, expected) + +@parametrize("x, expected", [(2, 4), (3, 7)]) +def test_double_failure(x: int, expected: int) -> None: + assert_eq(x * 2, expected) + +@skipif(platform() == "{platform}", reason="host platform") +def test_skip_on_platform_probe() -> None: + assert_eq(1, 0) + +@xfailif(feature("known_bug"), reason="feature-gated known issue") +def test_feature_xfail() -> None: + assert_eq(1, 0) + +@timeout("1ms") +def test_timeout_marker() -> None: + sleep(Duration.from_millis(100)) +"# + ), + ); + + let strict_smoke = run_incan_test_with_args(&dir, &["--list", "-m", "smoke", "--strict-markers"]); + let strict_smoke_stdout = String::from_utf8_lossy(&strict_smoke.stdout); + let strict_smoke_stderr = String::from_utf8_lossy(&strict_smoke.stderr); + assert!( + strict_smoke.status.success(), + "expected strict registered marker list to succeed.\nstdout:\n{}\nstderr:\n{}", + strict_smoke_stdout, + strict_smoke_stderr, + ); + assert!(strict_smoke_stdout.contains("test_runner_collection_surface.incn::test_inherited_smoke")); + + let strict_error = run_incan_test_with_args(&dir, &["--list", "-m", "missing", "--strict-markers"]); + let strict_stderr = String::from_utf8_lossy(&strict_error.stderr); + assert!( + !strict_error.status.success(), + "expected unknown strict marker to fail.\nstderr:\n{}", + strict_stderr, ); + assert!(strict_stderr.contains("unknown marker `missing`")); - let output = run_incan_test_with_args( + let marker_list = run_incan_test_with_args( &dir, &["--list", "-m", "api and not slow", "--strict-markers", "--slow"], ); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let marker_stdout = String::from_utf8_lossy(&marker_list.stdout); + let marker_stderr = String::from_utf8_lossy(&marker_list.stderr); assert!( - output.status.success(), + marker_list.status.success(), "expected boolean marker expression to collect.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!(stdout.contains("test_marker_expr.incn::test_api")); - assert!(!stdout.contains("test_marker_expr.incn::test_api_slow")); - assert!(!stdout.contains("test_marker_expr.incn::test_db")); - - let invalid = run_incan_test_with_args(&dir, &["--list", "-m", "api and ("]); - let invalid_stderr = String::from_utf8_lossy(&invalid.stderr); - assert!( - !invalid.status.success(), - "expected invalid marker expression to fail.\nstderr:\n{}", - invalid_stderr, - ); - assert!(invalid_stderr.contains("expected marker name or parenthesized expression")); - Ok(()) - } - - #[test] - fn e2e_slow_marker_is_excluded_by_default_and_included_with_flag() { - let dir = write_test_project( - "test_slow_filter.incn", - r#" -from std.testing import assert_eq, slow - -def test_fast() -> None: - assert_eq(1, 1) - -@slow -def test_slow_case() -> None: - assert_eq(1, 1) -"#, + marker_stdout, + marker_stderr, ); + assert!(marker_stdout.contains("test_runner_collection_surface.incn::test_api")); + assert!(!marker_stdout.contains("test_runner_collection_surface.incn::test_api_slow")); + assert!(!marker_stdout.contains("test_runner_collection_surface.incn::test_db")); let default_list = run_incan_test_with_args(&dir, &["--list"]); let default_stdout = String::from_utf8_lossy(&default_list.stdout); @@ -7890,8 +7887,8 @@ def test_slow_case() -> None: "expected default list to succeed.\nstdout:\n{}", default_stdout, ); - assert!(default_stdout.contains("test_slow_filter.incn::test_fast")); - assert!(!default_stdout.contains("test_slow_filter.incn::test_slow_case")); + assert!(default_stdout.contains("test_runner_collection_surface.incn::test_fast")); + assert!(!default_stdout.contains("test_runner_collection_surface.incn::test_slow_case")); let slow_list = run_incan_test_with_args(&dir, &["--list", "--slow"]); let slow_stdout = String::from_utf8_lossy(&slow_list.stdout); @@ -7900,82 +7897,93 @@ def test_slow_case() -> None: "expected --slow list to succeed.\nstdout:\n{}", slow_stdout, ); - assert!(slow_stdout.contains("test_slow_filter.incn::test_fast")); - assert!(slow_stdout.contains("test_slow_filter.incn::test_slow_case")); - } + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_fast")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_slow_case")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_marked_double[one-three]")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_marked_double[two-four]")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_pair[one-ten]")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_pair[one-twenty]")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_pair[two-ten]")); + assert!(slow_stdout.contains("test_runner_collection_surface.incn::test_pair[two-twenty]")); - #[test] - fn e2e_parametrize_case_ids_and_marks_affect_collection() { - let dir = write_test_project( - "test_case_ids.incn", - r#" -from std.testing import assert_eq, param_case, parametrize, xfail + let marked_run = run_incan_test_with_args(&dir, &["-k", "test_marked_double"]); + let marked_stdout = String::from_utf8_lossy(&marked_run.stdout); + let marked_stderr = String::from_utf8_lossy(&marked_run.stderr); + assert!( + marked_run.status.success(), + "expected xfailed case and passing case to make the run succeed.\nstdout:\n{}\nstderr:\n{}", + marked_stdout, + marked_stderr, + ); + assert!(marked_stdout.contains("xfailed") || marked_stdout.contains("XFAIL")); -@parametrize("x, expected", [ - param_case((1, 3), marks=[xfail("known")], id="one-three"), - (2, 4), -], ids=["ignored", "two-four"]) -def test_double(x: int, expected: int) -> None: - assert_eq(x * 2, expected) -"#, + let add_run = run_incan_test_with_args(&dir, &["--verbose", "-k", "test_add"]); + let add_stdout = String::from_utf8_lossy(&add_run.stdout); + let add_stderr = String::from_utf8_lossy(&add_run.stderr); + assert!( + add_run.status.success(), + "expected parametrized test to succeed.\nstdout:\n{}\nstderr:\n{}", + add_stdout, + add_stderr, ); + assert!(add_stdout.contains("test_add[1-2-3]")); + assert!(add_stdout.contains("test_add[10-20-30]")); + assert!(add_stdout.contains("test_add[0-0-0]")); + assert!(add_stdout.contains("3 passed")); - let listed = run_incan_test_with_args(&dir, &["--list"]); - let stdout = String::from_utf8_lossy(&listed.stdout); - let stderr = String::from_utf8_lossy(&listed.stderr); + let failing_param = run_incan_test_with_args(&dir, &["--verbose", "-k", "test_double_failure"]); + let failing_param_stdout = String::from_utf8_lossy(&failing_param.stdout); assert!( - listed.status.success(), - "expected parametrized list to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + !failing_param.status.success(), + "expected one failing case to make the run fail.\nstdout:\n{}", + failing_param_stdout, ); - assert!(stdout.contains("test_case_ids.incn::test_double[one-three]")); - assert!(stdout.contains("test_case_ids.incn::test_double[two-four]")); + assert!(failing_param_stdout.contains("1 passed") && failing_param_stdout.contains("1 failed")); - let run = run_incan_test(&dir); - let run_stdout = String::from_utf8_lossy(&run.stdout); - let run_stderr = String::from_utf8_lossy(&run.stderr); + let skip_run = run_incan_test_with_args(&dir, &["-k", "test_skip_on_platform_probe"]); + let skip_stdout = String::from_utf8_lossy(&skip_run.stdout); + let skip_stderr = String::from_utf8_lossy(&skip_run.stderr); assert!( - run.status.success(), - "expected xfailed case and passing case to make the run succeed.\nstdout:\n{}\nstderr:\n{}", - run_stdout, - run_stderr, + skip_run.status.success(), + "expected skipif probe to make the run successful.\nstdout:\n{}\nstderr:\n{}", + skip_stdout, + skip_stderr, ); - assert!(run_stdout.contains("xfailed") || run_stdout.contains("XFAIL")); - } + assert!(skip_stdout.contains("SKIPPED") || skip_stdout.contains("skipped")); - #[test] - fn e2e_stacked_parametrize_lists_cartesian_product_ids() { - let dir = write_test_project( - "test_parametrize_product.incn", - r#" -from std.testing import assert_eq, parametrize + let without_feature = run_incan_test_with_args(&dir, &["-k", "test_feature_xfail"]); + let without_stdout = String::from_utf8_lossy(&without_feature.stdout); + let without_stderr = String::from_utf8_lossy(&without_feature.stderr); + assert!( + !without_feature.status.success(), + "expected feature-gated xfail to run as an ordinary failing test without --feature.\nstdout:\n{}\nstderr:\n{}", + without_stdout, + without_stderr, + ); -@parametrize("x", [1, 2], ids=["one", "two"]) -@parametrize("y", [10, 20], ids=["ten", "twenty"]) -def test_pair(x: int, y: int) -> None: - assert_eq(x < y, true) -"#, + let with_feature = run_incan_test_with_args(&dir, &["--feature", "known_bug", "-k", "test_feature_xfail"]); + let with_feature_stdout = String::from_utf8_lossy(&with_feature.stdout); + let with_feature_stderr = String::from_utf8_lossy(&with_feature.stderr); + assert!( + with_feature.status.success(), + "expected xfailif probe to make the run successful.\nstdout:\n{}\nstderr:\n{}", + with_feature_stdout, + with_feature_stderr, ); + assert!(with_feature_stdout.contains("XFAIL") || with_feature_stdout.contains("xfailed")); - let listed = run_incan_test_with_args(&dir, &["--list"]); - let stdout = String::from_utf8_lossy(&listed.stdout); - let stderr = String::from_utf8_lossy(&listed.stderr); + let timeout = run_incan_test_with_args(&dir, &["-k", "test_timeout_marker"]); + let timeout_stdout = String::from_utf8_lossy(&timeout.stdout); + let timeout_stderr = String::from_utf8_lossy(&timeout.stderr); assert!( - listed.status.success(), - "expected stacked parametrized list to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + !timeout.status.success(), + "expected timeout marker to fail the test.\nstdout:\n{}\nstderr:\n{}", + timeout_stdout, + timeout_stderr, ); - assert!(stdout.contains("test_parametrize_product.incn::test_pair[one-ten]")); - assert!(stdout.contains("test_parametrize_product.incn::test_pair[one-twenty]")); - assert!(stdout.contains("test_parametrize_product.incn::test_pair[two-ten]")); - assert!(stdout.contains("test_parametrize_product.incn::test_pair[two-twenty]")); - } + assert!(timeout_stdout.contains("timed out after")); - #[test] - fn e2e_parametrize_arity_mismatch_is_collection_error() { - let dir = write_test_project( + let arity_dir = write_test_project( "test_parametrize_arity.incn", r#" from std.testing import parametrize @@ -7985,91 +7993,28 @@ def test_bad_case(x: int, y: int) -> None: pass "#, ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let arity_output = run_incan_test(&arity_dir); + let arity_stdout = String::from_utf8_lossy(&arity_output.stdout); + let arity_stderr = String::from_utf8_lossy(&arity_output.stderr); assert!( - !output.status.success(), + !arity_output.status.success(), "expected arity mismatch to fail during collection.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!(stderr.contains("parametrize case `1`")); - assert!(stderr.contains("expected 2 value(s)")); - } - - #[test] - fn e2e_timeout_marks_slow_test_failed() { - let dir = write_test_project( - "test_timeout.incn", - r#" -from rust::std::thread import sleep -from rust::std::time import Duration - -def test_slow() -> None: - sleep(Duration.from_millis(100)) -"#, - ); - - let output = run_incan_test_with_args(&dir, &["--timeout", "1ms"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - !output.status.success(), - "expected timeout run to fail.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!(stdout.contains("timed out after")); - } - - #[test] - fn e2e_conditional_markers_evaluate_collection_probes() { - let platform = std::env::consts::OS; - let dir = write_test_project( - "test_conditional_markers.incn", - &format!( - r#" -from std.testing import assert_eq, feature, platform, skipif, xfailif - -@skipif(platform() == "{platform}", reason="host platform") -def test_skip_on_platform_probe() -> None: - assert_eq(1, 0) - -@xfailif(feature("known_bug"), reason="feature-gated known issue") -def test_feature_xfail() -> None: - assert_eq(1, 0) -"# - ), - ); - - let without_feature = run_incan_test_with_args(&dir, &["-k", "test_feature_xfail"]); - let without_stdout = String::from_utf8_lossy(&without_feature.stdout); - let without_stderr = String::from_utf8_lossy(&without_feature.stderr); - assert!( - !without_feature.status.success(), - "expected feature-gated xfail to run as an ordinary failing test without --feature.\nstdout:\n{}\nstderr:\n{}", - without_stdout, - without_stderr, + arity_stdout, + arity_stderr, ); + assert!(arity_stderr.contains("parametrize case `1`")); + assert!(arity_stderr.contains("expected 2 value(s)")); - let output = run_incan_test_with_args(&dir, &["--feature", "known_bug"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let invalid_marker = run_incan_test_with_args(&dir, &["--list", "-m", "api and ("]); + let invalid_marker_stderr = String::from_utf8_lossy(&invalid_marker.stderr); assert!( - output.status.success(), - "expected skipif/xfailif probes to make the run successful.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + !invalid_marker.status.success(), + "expected invalid marker expression to fail.\nstderr:\n{}", + invalid_marker_stderr, ); - assert!(stdout.contains("SKIPPED") || stdout.contains("skipped")); - assert!(stdout.contains("XFAIL") || stdout.contains("xfailed")); - } + assert!(invalid_marker_stderr.contains("expected marker name or parenthesized expression")); - #[test] - fn e2e_conditional_marker_rejects_runtime_expression() { - let dir = write_test_project( + let bad_conditional_dir = write_test_project( "test_bad_conditional_marker.incn", r#" from std.testing import skipif @@ -8083,7 +8028,7 @@ def test_dynamic_condition() -> None: "#, ); - let output = run_incan_test(&dir); + let output = run_incan_test(&bad_conditional_dir); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( @@ -8108,7 +8053,7 @@ from rust::std::thread import sleep from rust::std::time import Duration def test_sleep_a() -> None: - sleep(Duration.from_millis(1200)) + sleep(Duration.from_millis(600)) "#, ); let second = dir.join("test_sleep_b.incn"); @@ -8119,25 +8064,11 @@ from rust::std::thread import sleep from rust::std::time import Duration def test_sleep_b() -> None: - sleep(Duration.from_millis(1200)) + sleep(Duration.from_millis(600)) "#, )?; - let sequential_start = std::time::Instant::now(); - let sequential = run_incan_test_with_args(&dir, &["--jobs", "1"]); - let sequential_elapsed = sequential_start.elapsed(); - let sequential_stdout = String::from_utf8_lossy(&sequential.stdout); - let sequential_stderr = String::from_utf8_lossy(&sequential.stderr); - assert!( - sequential.status.success(), - "expected sequential warm-up run to pass.\nstdout:\n{}\nstderr:\n{}", - sequential_stdout, - sequential_stderr, - ); - - let parallel_start = std::time::Instant::now(); let parallel = run_incan_test_with_args(&dir, &["--jobs", "2"]); - let parallel_elapsed = parallel_start.elapsed(); let parallel_stdout = String::from_utf8_lossy(¶llel.stdout); let parallel_stderr = String::from_utf8_lossy(¶llel.stderr); assert!( @@ -8146,11 +8077,22 @@ def test_sleep_b() -> None: parallel_stdout, parallel_stderr, ); - assert!( - parallel_elapsed + std::time::Duration::from_millis(500) < sequential_elapsed, - "expected --jobs 2 to run independent file batches concurrently; sequential={:?}, parallel={:?}\nparallel stdout:\n{}", - sequential_elapsed, - parallel_elapsed, + let running_a = parallel_stdout + .find("test_sleep_a.incn (1 item(s))") + .ok_or("expected parallel output to announce test_sleep_a.incn")?; + let running_b = parallel_stdout + .find("test_sleep_b.incn (1 item(s))") + .ok_or("expected parallel output to announce test_sleep_b.incn")?; + let passed_a = parallel_stdout + .find("test_sleep_a.incn::test_sleep_a PASSED") + .ok_or("expected parallel output to report test_sleep_a passing")?; + let passed_b = parallel_stdout + .find("test_sleep_b.incn::test_sleep_b PASSED") + .ok_or("expected parallel output to report test_sleep_b passing")?; + let first_pass = passed_a.min(passed_b); + assert!( + running_a < first_pass && running_b < first_pass, + "expected --jobs 2 to launch both independent file batches before either completed\nparallel stdout:\n{}", parallel_stdout, ); Ok(()) @@ -8175,7 +8117,7 @@ from rust::std::thread import sleep from rust::std::time import Duration def test_b_slow() -> None: - sleep(Duration.from_millis(3000)) + sleep(Duration.from_millis(800)) "#, )?; let warmup = run_incan_test_with_args(&dir, &["--jobs", "1", "-k", "test_b_slow"]); @@ -8221,7 +8163,7 @@ from std.testing import resource @resource("db") def test_resource_a() -> None: - sleep(Duration.from_millis(1000)) + sleep(Duration.from_millis(700)) "#, ); std::fs::write( @@ -8233,7 +8175,7 @@ from std.testing import resource @resource("db") def test_resource_b() -> None: - sleep(Duration.from_millis(1000)) + sleep(Duration.from_millis(700)) "#, )?; @@ -8259,7 +8201,7 @@ def test_resource_b() -> None: stderr, ); assert!( - elapsed >= std::time::Duration::from_millis(1800), + elapsed >= std::time::Duration::from_millis(1200), "expected shared @resource workers not to overlap; elapsed={:?}\nstdout:\n{}", elapsed, stdout, @@ -8278,7 +8220,7 @@ from std.testing import serial @serial def test_serial() -> None: - sleep(Duration.from_millis(1000)) + sleep(Duration.from_millis(700)) "#, ); std::fs::write( @@ -8288,7 +8230,7 @@ from rust::std::thread import sleep from rust::std::time import Duration def test_regular() -> None: - sleep(Duration.from_millis(1000)) + sleep(Duration.from_millis(700)) "#, )?; @@ -8314,7 +8256,7 @@ def test_regular() -> None: stderr, ); assert!( - elapsed >= std::time::Duration::from_millis(1800), + elapsed >= std::time::Duration::from_millis(1200), "expected @serial worker to run alone; elapsed={:?}\nstdout:\n{}", elapsed, stdout, @@ -8323,29 +8265,7 @@ def test_regular() -> None: } #[test] - fn e2e_nocapture_prints_passing_test_output() { - let dir = write_test_project( - "test_capture.incn", - r#" -def test_prints() -> None: - print("VISIBLE_CAPTURE") -"#, - ); - - let output = run_incan_test_with_args(&dir, &["--nocapture"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected nocapture run to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!(stdout.contains("VISIBLE_CAPTURE")); - } - - #[test] - fn e2e_sequential_single_file_runs_do_not_cross_wire_relative_paths() { + fn e2e_sequential_single_file_runs_do_not_cross_wire_paths() { let dir = write_test_project( "incan.toml", r#"[project] @@ -8419,18 +8339,15 @@ def test_beta_only() -> None: "expected no missing-outcome diagnostic in second run.\noutput:\n{}", second_combined, ); - } - #[test] - fn e2e_sequential_single_file_runs_do_not_cross_wire_absolute_paths() { - let dir = write_test_project( + let abs_dir = write_test_project( "incan.toml", r#"[project] name = "session_isolation_absolute" version = "0.1.0" "#, ); - let tests_dir = dir.join("tests"); + let tests_dir = abs_dir.join("tests"); if let Err(err) = std::fs::create_dir_all(&tests_dir) { panic!("failed to create tests dir: {}", err); } @@ -8559,7 +8476,7 @@ def test_nested_dataset_modules() -> None: } #[test] - fn e2e_test_runner_preserves_project_fixture_cwd_for_file_and_batch_runs() { + fn e2e_test_runner_preserves_fixture_cwd_for_file_and_batch_runs() { let dir = write_test_project( "incan.toml", r#"[project] @@ -8610,21 +8527,18 @@ def test_fixture_path_exists() -> None: batch_stdout, batch_stderr, ); - } - #[test] - fn e2e_test_runner_preserves_fixture_cwd_without_manifest_for_file_and_batch_runs() { use std::time::{SystemTime, UNIX_EPOCH}; - let mut dir = std::env::temp_dir(); + let mut bare_dir = std::env::temp_dir(); let Ok(duration) = SystemTime::now().duration_since(UNIX_EPOCH) else { panic!("system time before UNIX epoch"); }; - dir.push(format!("incan_e2e_test_nomani_{}", duration.as_nanos())); - if let Err(err) = std::fs::create_dir_all(&dir) { + bare_dir.push(format!("incan_e2e_test_nomani_{}", duration.as_nanos())); + if let Err(err) = std::fs::create_dir_all(&bare_dir) { panic!("failed to create temp dir: {}", err); } - let tests_dir = dir.join("tests"); + let tests_dir = bare_dir.join("tests"); let fixtures_dir = tests_dir.join("fixtures"); if let Err(err) = std::fs::create_dir_all(&fixtures_dir) { @@ -8650,7 +8564,7 @@ def test_cwd__fixture_path_is_repo_relative() -> None: panic!("failed to write fixture path test: {}", err); } - let single = run_incan_test_relative(&dir, "tests/test_cwd.incn"); + let single = run_incan_test_relative(&bare_dir, "tests/test_cwd.incn"); let single_stdout = String::from_utf8_lossy(&single.stdout); let single_stderr = String::from_utf8_lossy(&single.stderr); assert!( @@ -8660,7 +8574,7 @@ def test_cwd__fixture_path_is_repo_relative() -> None: single_stderr, ); - let batch = run_incan_test_relative(&dir, "tests"); + let batch = run_incan_test_relative(&bare_dir, "tests"); let batch_stdout = String::from_utf8_lossy(&batch.stdout); let batch_stderr = String::from_utf8_lossy(&batch.stderr); assert!( @@ -8672,80 +8586,167 @@ def test_cwd__fixture_path_is_repo_relative() -> None: } #[test] - fn e2e_imported_pub_static_scalar_read_in_tests_succeeds() { + fn e2e_inline_and_imported_surfaces_share_one_project() -> Result<(), Box> { let dir = write_test_project( "incan.toml", r#"[project] -name = "pub_static_scalar_read" +name = "inline_and_imported_surface_batch" version = "0.1.0" "#, ); let src_dir = dir.join("src"); let tests_dir = dir.join("tests"); + std::fs::create_dir_all(&src_dir)?; + std::fs::create_dir_all(&tests_dir)?; + std::fs::write(src_dir.join("widgets.incn"), "pub static MARKER: int = 41\n")?; + std::fs::write( + src_dir.join("defaults.incn"), + r#" +pub def fallback() -> int: + return 2 +"#, + )?; + std::fs::write( + src_dir.join("helper.incn"), + r#" +from defaults import fallback - if let Err(err) = std::fs::create_dir_all(&src_dir) { - panic!("failed to create src dir: {}", err); - } - if let Err(err) = std::fs::create_dir_all(&tests_dir) { - panic!("failed to create tests dir: {}", err); - } - if let Err(err) = std::fs::write(src_dir.join("widgets.incn"), "pub static MARKER: int = 41\n") { - panic!("failed to write widgets source: {}", err); - } - if let Err(err) = std::fs::write( - tests_dir.join("test_widgets_static.incn"), +pub def combine(left: int, middle: int = fallback(), right: int = 3) -> int: + return left + middle + right +"#, + )?; + std::fs::write( + src_dir.join("helpers.incn"), + r#" +pub def count_names(names: List[str]) -> int: + return len(names) +"#, + )?; + std::fs::write( + src_dir.join("registry.incn"), + r#" +pub const TOKEN: str = "token" +pub const DECORATOR_TOKEN: str = "probe.value" + +def keep_int(func: (int) -> int) -> (int) -> int: + return func + +pub def registered(_name: str) -> Callable[(int) -> int, (int) -> int]: + return keep_int +"#, + )?; + let entry = src_dir.join("main.incn"); + std::fs::write( + &entry, + r#" +def add(a: int, b: int) -> int: + return a + b + +def secret() -> str: + return "private" + +def main() -> None: + println("production") + +module tests: + from rust::incan_stdlib::testing import TestEnv + from rust::std::path import PathBuf + import std.testing as testing + from std.testing import assert_eq, assert_is_some, fixture, test + + @fixture(autouse=true) + def seed() -> int: + return 40 + + @fixture + def answer(seed: int) -> int: + return seed + 2 + + def test_inline_addition(seed: int) -> None: + assert_eq(seed, 40) + assert_eq(add(2, 3), 5) + + def test_inline_private_access(seed: int) -> None: + assert_eq(seed, 40) + assert_eq(secret(), "private") + + def test_inline_assert_helper(seed: int) -> None: + assert_eq(seed, 40) + testing.assert(True) + + @test + def decorated_inline_case(seed: int) -> None: + assert_eq(seed, 40) + assert_eq(add(20, 22), 42) + + def test_inline_fixture_and_tmp_path(answer: int, tmp_path: PathBuf) -> None: + assert_eq(answer, 42) + assert_eq(tmp_path.exists(), true) + + def test_inline_tmp_workdir(tmp_workdir: PathBuf) -> None: + assert_eq(tmp_workdir.exists(), true) + + def test_inline_env_fixture(mut env: TestEnv) -> None: + env.set("INCAN_INLINE_ENV_FIXTURE", "set") + assert_eq(assert_is_some(env.get("INCAN_INLINE_ENV_FIXTURE")), "set") + env.unset("INCAN_INLINE_ENV_FIXTURE") + assert_eq(env.get("INCAN_INLINE_ENV_FIXTURE"), None) +"#, + )?; + std::fs::write( + tests_dir.join("test_imported_surface_batch.incn"), r#" from std.testing import assert_eq +from helper import combine +from helpers import count_names +from registry import DECORATOR_TOKEN, TOKEN, registered from widgets import MARKER +def identity(value: str) -> str: + return value + +@registered(DECORATOR_TOKEN) +def increment(value: int) -> int: + return value + 1 + +def test_imported_const_str_call_arguments_materialize() -> None: + local: str = TOKEN + assert_eq(identity(TOKEN), "token") + assert_eq(identity(TOKEN.to_string()), "token") + assert_eq(identity(local), "token") + assert_eq(TOKEN.upper(), "TOKEN") + +def test_imported_decorator_factory_const_str_argument_materializes() -> None: + assert_eq(increment(1), 2) + def test_imported_pub_static_scalar_read() -> None: assert_eq(MARKER, 41) -"#, - ) { - panic!("failed to write widget static test: {}", err); - } - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected imported pub static scalar read test to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - } +def test_empty_names() -> None: + assert_eq(count_names([]), 0) - #[test] - fn e2e_empty_list_arguments_in_tests_preserve_string_element_type() -> Result<(), Box> { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "empty_list_test" -version = "0.1.0" -"#, - ); - let src_dir = dir.join("src"); - let tests_dir = dir.join("tests"); +def test_assert_statement_sugar() -> None: + assert 1 + 1 == 2 + assert 3 != 4 + assert not False + assert True - std::fs::create_dir_all(&src_dir)?; - std::fs::create_dir_all(&tests_dir)?; - std::fs::write( - src_dir.join("helpers.incn"), - r#" -pub def count_names(names: List[str]) -> int: - return len(names) +def test_imported_default_expression_expands_with_required_imports() -> None: + assert_eq(combine(left=1, right=4), 7, "default expression helper should be available after expansion") "#, )?; + let production_entry = src_dir.join("production_only.incn"); std::fs::write( - tests_dir.join("test_empty_names.incn"), + &production_entry, r#" -from std.testing import assert_eq -from helpers import count_names +def main() -> None: + println("production") -def test_empty_names() -> None: - assert_eq(count_names([]), 0) +module tests: + from std.testing import assert_eq + + def test_production() -> None: + assert_eq(1 + 1, 2) "#, )?; @@ -8755,10 +8756,27 @@ def test_empty_names() -> None: assert!( output.status.success(), - "expected empty list string arg test to succeed.\nstdout:\n{}\nstderr:\n{}", + "expected batched inline/imported test-runner surfaces to succeed.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); + assert!( + stdout.contains("main.incn::test_inline_addition") + && stdout.contains("main.incn::test_inline_private_access") + && stdout.contains("main.incn::decorated_inline_case") + && stdout.contains("main.incn::test_inline_fixture_and_tmp_path") + && stdout.contains("test_imported_surface_batch.incn::test_imported_pub_static_scalar_read") + && stdout.contains( + "test_imported_surface_batch.incn::test_imported_default_expression_expands_with_required_imports" + ), + "expected representative batched inline/imported test names.\nstdout:\n{}", + stdout + ); + assert!( + !stderr.contains("str_as_str") && !stderr.contains("expected `String`, found `&str`"), + "imported const str call and decorator arguments should materialize as owned strings.\nstderr:\n{}", + stderr, + ); assert!( !stderr.contains("type annotations needed"), "expected no Rust inference failure for empty string list.\nstderr:\n{}", @@ -8770,151 +8788,100 @@ def test_empty_names() -> None: stderr, ); - Ok(()) - } - - #[test] - fn e2e_assert_statement_with_module_import_succeeds() { - let dir = write_test_project( - "test_assert_stmt.incn", - r#" -import std.testing - -def test_assert_statement_sugar() -> None: - assert 1 + 1 == 2 - assert 3 != 4 - assert not False - assert True -"#, - ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - + let listed = run_incan_test_with_args(&dir, &["--list", "-k", "decorated_inline_case"]); + let listed_stdout = String::from_utf8_lossy(&listed.stdout); + let listed_stderr = String::from_utf8_lossy(&listed.stderr); assert!( - output.status.success(), - "expected assert-statement test to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + listed.status.success(), + "expected inline --list -k run to succeed.\nstdout:\n{}\nstderr:\n{}", + listed_stdout, + listed_stderr, ); assert!( - stdout.contains("PASSED") || stdout.contains("passed"), - "expected PASSED in output.\nstdout:\n{}", - stdout, + listed_stdout + .lines() + .any(|line| line == "src/main.incn::decorated_inline_case"), + "expected decorated inline test id in --list output.\nstdout:\n{}", + listed_stdout, ); - } - - #[test] - fn e2e_inline_module_tests_are_discovered_and_run() -> Result<(), Box> { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "inline_module_tests_run" -version = "0.1.0" -"#, + assert!( + !listed_stdout.contains("src/main.incn::test_inline_addition"), + "expected keyword filter to hide the name-discovered inline test.\nstdout:\n{}", + listed_stdout, ); - let src_dir = dir.join("src"); - std::fs::create_dir_all(&src_dir)?; - std::fs::write( - src_dir.join("main.incn"), - r#" -def add(a: int, b: int) -> int: - return a + b - -def main() -> None: - pass - -module tests: - from std.testing import assert_eq - - def test_addition() -> None: - assert_eq(add(2, 3), 5) -"#, - )?; - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let out_dir = dir.join("out"); + let build_output = run_incan_build(&production_entry, &out_dir); + let build_stderr = String::from_utf8_lossy(&build_output.stderr); assert!( - output.status.success(), - "expected inline module test run to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + build_output.status.success(), + "expected production build to ignore inline test imports.\nstderr:\n{}", + build_stderr, ); + let main_rs = std::fs::read_to_string(out_dir.join("src/main.rs"))?; assert!( - stdout.contains("1 passed"), - "expected inline test to run.\nstdout:\n{}", - stdout + !main_rs.contains("__incan_std::testing"), + "inline test import should not leak into generated production code:\n{}", + main_rs, + ); + assert!( + !main_rs.contains("test_inline_addition"), + "inline test function should not leak into generated production code:\n{}", + main_rs, ); Ok(()) } #[test] - fn e2e_inline_module_tests_can_access_private_enclosing_names() -> Result<(), Box> { + fn e2e_imported_generic_decorator_factory_preserves_function_signatures() -> Result<(), Box> + { let dir = write_test_project( "incan.toml", r#"[project] -name = "inline_private_access" +name = "generic_decorator_factory" version = "0.1.0" "#, ); let src_dir = dir.join("src"); + let tests_dir = dir.join("tests"); std::fs::create_dir_all(&src_dir)?; + std::fs::create_dir_all(&tests_dir)?; std::fs::write( - src_dir.join("main.incn"), + src_dir.join("registry.incn"), r#" -def secret() -> str: - return "private" - -def main() -> None: - pass - -module tests: - from std.testing import assert_eq - - def test_secret() -> None: - assert_eq(secret(), "private") +pub def registered[F](name: str) -> ((F) -> F): + return (func) => func "#, )?; + std::fs::write( + src_dir.join("columns.incn"), + r#" +from registry import registered - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); +pub model ColumnExpr: + pub name: str - assert!( - output.status.success(), - "expected inline module test to access enclosing private helper.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - Ok(()) - } +@registered[(str) -> ColumnExpr]("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) - #[test] - fn e2e_inline_module_std_testing_assert_helper_is_normalized_before_codegen() - -> Result<(), Box> { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "inline_assert_helper" -version = "0.1.0" +@registered("inql.functions.literal") +pub def literal() -> ColumnExpr: + return ColumnExpr(name="literal") "#, - ); - let src_dir = dir.join("src"); - std::fs::create_dir_all(&src_dir)?; + )?; std::fs::write( - src_dir.join("main.incn"), + tests_dir.join("test_generic_decorator_factory.incn"), r#" -def main() -> None: - pass +from std.testing import assert_eq +from columns import col, literal -module tests: - import std.testing as testing +def test_explicit_generic_decorator_factory_signature() -> None: + assert_eq(col("id").name, "id") - def test_assert_helper() -> None: - testing.assert(True) +def test_inferred_generic_decorator_factory_signature() -> None: + assert_eq(literal().name, "literal") "#, )?; @@ -8923,194 +8890,139 @@ module tests: let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected inline one-argument std.testing.assert call to be normalized before codegen.\nstdout:\n{}\nstderr:\n{}", + "expected imported generic decorator factory project to pass.\nstdout:\n{}\nstderr:\n{}", stdout, - stderr + stderr, ); - assert!(stdout.contains("test_assert_helper")); Ok(()) } #[test] - fn e2e_inline_module_test_imports_do_not_affect_build() -> Result<(), Box> { + fn e2e_inline_decorated_sum_shadows_builtin_sum_issue677() -> Result<(), Box> { let dir = write_test_project( "incan.toml", r#"[project] -name = "inline_imports_do_not_affect_build" +name = "decorated_sum_inline" version = "0.1.0" "#, ); let src_dir = dir.join("src"); std::fs::create_dir_all(&src_dir)?; - let entry = src_dir.join("main.incn"); + let source_path = src_dir.join("functions.incn"); std::fs::write( - &entry, + &source_path, r#" -def main() -> None: - println("production") - -module tests: - from std.testing import assert_eq +pub model IntExpr: + pub value: int - def test_production() -> None: - assert_eq(1 + 1, 2) -"#, - )?; +pub model TextExpr: + pub value: str - let out_dir = dir.join("out"); - let output = run_incan_build(&entry, &out_dir); - let stderr = String::from_utf8_lossy(&output.stderr); +pub type Expr = IntExpr | TextExpr - assert!( - output.status.success(), - "expected production build to ignore inline test imports.\nstderr:\n{}", - stderr, - ); - let main_rs = std::fs::read_to_string(out_dir.join("src/main.rs"))?; - assert!( - !main_rs.contains("__incan_std::testing"), - "inline test import should not leak into generated production code:\n{}", - main_rs, - ); - assert!( - !main_rs.contains("test_production"), - "inline test function should not leak into generated production code:\n{}", - main_rs, - ); - Ok(()) - } +pub model Measure: + pub kind: str - #[test] - fn e2e_inline_module_test_decorator_list_and_keyword_filter() -> Result<(), Box> { - let dir = write_test_project( - "incan.toml", - r#"[project] -name = "inline_decorator_list_filter" -version = "0.1.0" -"#, - ); - let src_dir = dir.join("src"); - std::fs::create_dir_all(&src_dir)?; - std::fs::write( - src_dir.join("math.incn"), - r#" -def add(a: int, b: int) -> int: - return a + b +pub def registered[F](function_ref: str) -> ((F) -> F): + return (func) => func -module tests: - from std.testing import assert_eq, test +pub def expr(value: int) -> Expr: + return IntExpr(value=value) - @test - def checks_sum() -> None: - assert_eq(add(20, 22), 42) +@registered("demo.sum") +pub def sum(value: Expr) -> Measure: + return Measure(kind="local") - def test_by_name() -> None: - assert_eq(add(1, 1), 2) +module tests: + def test_inline_test_resolves_decorated_sum_before_builtin_sum() -> None: + measure = sum(expr(1)) + assert measure.kind == "local" "#, )?; - let output = run_incan_test_with_args(&dir, &["--list", "-k", "checks_sum"]); + let output = run_incan_test_path(&source_path); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); - - assert!( - output.status.success(), - "expected inline --list -k run to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); assert!( - stdout.lines().any(|line| line == "src/math.incn::checks_sum"), - "expected decorated inline test id in --list output.\nstdout:\n{}", + output.status.success(), + "expected decorated inline sum test to pass.\nstdout:\n{}\nstderr:\n{}", stdout, + stderr, ); assert!( - !stdout.contains("src/math.incn::test_by_name"), - "expected keyword filter to hide the name-discovered inline test.\nstdout:\n{}", + stdout.contains("functions.incn::test_inline_test_resolves_decorated_sum_before_builtin_sum"), + "expected the #677 inline test to run.\nstdout:\n{}\nstderr:\n{}", stdout, + stderr, ); Ok(()) } #[test] - fn e2e_inline_module_parametrize_markers_strict_and_timeout() -> Result<(), Box> { + fn e2e_conventional_test_batches_split_import_declaration_collisions_issue676() + -> Result<(), Box> { let dir = write_test_project( "incan.toml", r#"[project] -name = "inline_parametrize_markers" +name = "import_collision_batch" version = "0.1.0" "#, ); let src_dir = dir.join("src"); + let tests_dir = dir.join("tests"); std::fs::create_dir_all(&src_dir)?; + std::fs::create_dir_all(&tests_dir)?; std::fs::write( - src_dir.join("math.incn"), + src_dir.join("helpers.incn"), r#" -module tests: - from rust::std::thread import sleep - from rust::std::time import Duration - from std.testing import assert_eq, mark, param_case, parametrize, timeout, xfail - - const TEST_MARKERS: List[str] = ["smoke"] - const TEST_MARKS: List[str] = ["smoke"] - - @parametrize("x, expected", [ - param_case((1, 3), marks=[xfail("known")], id="one-three"), - (2, 4), - ], ids=["ignored", "two-four"]) - def test_double(x: int, expected: int) -> None: - assert_eq(x * 2, expected) +pub def col() -> int: + return 1 +"#, + )?; + std::fs::write( + tests_dir.join("test_imports_col.incn"), + r#" +from helpers import col - @mark("smoke") - @timeout("1ms") - def test_timeout_marker() -> None: - sleep(Duration.from_millis(100)) +def test_imported_col() -> None: + assert col() == 1 "#, )?; + std::fs::write( + tests_dir.join("test_declares_col.incn"), + r#" +def col() -> int: + return 2 - let listed = run_incan_test_with_args(&dir, &["--list", "-m", "smoke", "--strict-markers"]); - let listed_stdout = String::from_utf8_lossy(&listed.stdout); - let listed_stderr = String::from_utf8_lossy(&listed.stderr); - assert!( - listed.status.success(), - "expected inline strict marker list to succeed.\nstdout:\n{}\nstderr:\n{}", - listed_stdout, - listed_stderr, - ); - assert!(listed_stdout.contains("src/math.incn::test_double[one-three]")); - assert!(listed_stdout.contains("src/math.incn::test_double[two-four]")); - assert!(listed_stdout.contains("src/math.incn::test_timeout_marker")); +def test_local_col() -> None: + assert col() == 2 +"#, + )?; - let run = run_incan_test_with_args(&dir, &["-k", "test_double"]); - let run_stdout = String::from_utf8_lossy(&run.stdout); - let run_stderr = String::from_utf8_lossy(&run.stderr); + let output = run_incan_test(&dir); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); assert!( - run.status.success(), - "expected inline parametrized xfail/pass cases to succeed.\nstdout:\n{}\nstderr:\n{}", - run_stdout, - run_stderr, + output.status.success(), + "expected import/local declaration collision batch to split and pass.\nstdout:\n{}\nstderr:\n{}", + stdout, + stderr, ); - assert!(run_stdout.contains("XFAIL") || run_stdout.contains("xfailed")); - - let timeout = run_incan_test_with_args(&dir, &["-k", "test_timeout_marker"]); - let timeout_stdout = String::from_utf8_lossy(&timeout.stdout); - let timeout_stderr = String::from_utf8_lossy(&timeout.stderr); assert!( - !timeout.status.success(), - "expected inline timeout marker to fail the test.\nstdout:\n{}\nstderr:\n{}", - timeout_stdout, - timeout_stderr, + stdout.contains("test_imported_col") && stdout.contains("test_local_col"), + "expected both split test files to run.\nstdout:\n{}\nstderr:\n{}", + stdout, + stderr, ); - assert!(timeout_stdout.contains("timed out after")); Ok(()) } #[test] - fn e2e_inline_module_fixtures_builtins_and_autouse() -> Result<(), Box> { + fn e2e_method_call_decorator_factories_use_checked_receiver_lowering() -> Result<(), Box> { let dir = write_test_project( "incan.toml", r#"[project] -name = "inline_fixture_builtins" +name = "method_call_decorator_factories" version = "0.1.0" "#, ); @@ -9119,286 +9031,455 @@ version = "0.1.0" std::fs::write( src_dir.join("main.incn"), r#" -module tests: - from rust::incan_stdlib::testing import TestEnv - from rust::std::path import PathBuf - from std.testing import assert_eq, assert_is_some, fixture +class Registry: + pub names: list[str] - @fixture(autouse=true) - def seed() -> int: - return 40 + @staticmethod + def new() -> Self: + return Registry(names=[]) - @fixture - def answer(seed: int) -> int: - return seed + 2 + @staticmethod + def add_static[F](name: str) -> (F) -> F: + FUNCTIONS.names.append(name) + return (func) => func - def test_fixture_and_tmp_path(answer: int, tmp_path: PathBuf) -> None: - assert_eq(answer, 42) - assert_eq(tmp_path.exists(), true) + def add[F](mut self, name: str) -> (F) -> F: + self.names.append(name) + return (func) => func - def test_tmp_workdir(tmp_workdir: PathBuf) -> None: - assert_eq(tmp_workdir.exists(), true) - def test_env_fixture(mut env: TestEnv) -> None: - env.set("INCAN_INLINE_ENV_FIXTURE", "set") - assert_eq(assert_is_some(env.get("INCAN_INLINE_ENV_FIXTURE")), "set") - env.unset("INCAN_INLINE_ENV_FIXTURE") - assert_eq(env.get("INCAN_INLINE_ENV_FIXTURE"), None) +static FUNCTIONS: Registry = Registry.new() + + +@Registry::add_static("static") +def static_col(name: str) -> str: + return name + + +@FUNCTIONS.add("instance") +def instance_col(name: str) -> str: + return name + + +def main() -> None: + println(static_col("amount")) + println(instance_col("price")) + println(len(FUNCTIONS.names)) "#, )?; - let output = run_incan_test(&dir); + let out_dir = dir.join("out"); + let output = run_incan_build(&src_dir.join("main.incn"), &out_dir); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected inline fixtures and built-ins to succeed.\nstdout:\n{}\nstderr:\n{}", + "expected method-call decorator factories to build.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); + + let generated = std::fs::read_to_string(out_dir.join("src/main.rs"))?; assert!( - stdout.contains("test_fixture_and_tmp_path"), - "expected inline fixture test name in output.\nstdout:\n{}", - stdout, + generated.contains("Registry :: add_static") + || generated.contains("Registry::add_static") + || generated.contains("Registry :: add_static ::"), + "class static method decorator should lower as associated function syntax:\n{}", + generated, ); assert!( - stdout.contains("test_tmp_workdir"), - "expected inline tmp_workdir test name in output.\nstdout:\n{}", - stdout, + generated.contains(".with_mut(|__incan_static_value|") + && (generated.contains("let __incan_static_arg_0 = \"instance\".to_string();") + || generated.contains("let __incan_static_arg_0 = \"instance\".into();")) + && generated.contains("__incan_static_value.add(__incan_static_arg_0)"), + "static registry receiver should lower through static storage access:\n{}", + generated, ); Ok(()) } #[test] - fn e2e_module_scoped_fixture_is_reused_within_file() -> Result<(), Box> { + fn build_lib_imported_static_decorator_receiver_materializes_string_arg_issue671() + -> Result<(), Box> { let dir = write_test_project( - "test_module_scope_fixture.incn", + "incan.toml", + r#"[project] +name = "imported_static_decorator_receiver" +version = "0.1.0" +"#, + ); + let src_dir = dir.join("src"); + std::fs::create_dir_all(&src_dir)?; + std::fs::write( + src_dir.join("probe_registry.incn"), r#" -from std.testing import assert_eq, fixture +@derive(Clone) +pub class ProbeRegistry: + @staticmethod + def new() -> Self: + return ProbeRegistry() -static calls: int = 0 + def add[F](mut self, name: str, value: int) -> (F) -> F: + return (func) => func -@fixture(scope="module") -def once() -> int: - calls += 1 - return calls -def test_first(once: int) -> None: - assert_eq(once, 1) +pub static PROBE_REGISTRY: ProbeRegistry = ProbeRegistry.new() +"#, + )?; + std::fs::write( + src_dir.join("probe_decorated.incn"), + r#" +from probe_registry import PROBE_REGISTRY -def test_second(once: int) -> None: - assert_eq(once, 1) +@PROBE_REGISTRY.add("decorated", 1) +pub def decorated(value: int) -> int: + return value "#, - ); + )?; + std::fs::write(src_dir.join("lib.incn"), "pub from probe_decorated import decorated\n")?; - let output = run_incan_test(&dir); + let output = incan_command() + .args(["build", "--lib"]) + .current_dir(&*dir) + .env("CARGO_NET_OFFLINE", "true") + .output()?; let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected module-scoped fixture value to be reused across tests in the same file.\nstdout:\n{}\nstderr:\n{}", + "expected imported static decorator receiver project to build for #671.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); - assert!(stdout.contains("test_first") && stdout.contains("test_second")); + + let generated = std::fs::read_to_string(dir.join("target/lib/src/probe_decorated.rs"))?; + assert!( + (generated.contains("let __incan_static_arg_0 = \"decorated\".into();") + || generated.contains("let __incan_static_arg_0 = \"decorated\".to_string();")) + && !generated.contains("__incan_static_arg_0.clone()"), + "imported static decorator string argument should materialize as owned String:\n{}", + generated, + ); Ok(()) } #[test] - fn e2e_yield_fixture_teardown_runs_after_failure() -> Result<(), Box> { + fn build_static_receiver_option_model_lookup_issue674() -> Result<(), Box> { let dir = write_test_project( - "test_yield_fixture_teardown.incn", + "main.incn", r#" -from std.testing import assert_eq, fixture +@derive(Clone) +model Entry: + value: int -static calls: int = 0 -@fixture -def resource() -> int: - calls += 1 - yield calls - calls += 10 +@derive(Clone) +class Registry: + entries: list[Entry] -def test_1_fails(resource: int) -> None: - assert_eq(resource, 99) + @staticmethod + def new() -> Self: + return Registry(entries=[Entry(value=1)]) -def test_2_observes_teardown() -> None: - assert_eq(calls, 11) + def entry(self, name: str) -> Option[Entry]: + if len(self.entries) == 0: + return None + return Some(self.entries[0]) + + +static REGISTRY: Registry = Registry.new() + + +pub def lookup() -> int: + match REGISTRY.entry("decorated"): + Some(entry) => return entry.value + None => return 0 + + +def main() -> None: + println(lookup()) "#, ); - let output = run_incan_test(&dir); + let out_dir = dir.join("out"); + let output = run_incan_build(&dir.join("main.incn"), &out_dir); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( - !output.status.success(), - "expected the intentionally failing test to fail the run.\nstdout:\n{}\nstderr:\n{}", + output.status.success(), + "expected static receiver Option model lookup to build for #674.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); + + let generated = std::fs::read_to_string(out_dir.join("src/main.rs"))?; assert!( - stdout.contains("test_2_observes_teardown PASSED"), - "expected teardown to run before the following test observed shared state.\nstdout:\n{}", - stdout, + generated.contains("match {\n let __incan_static_arg_0 = \"decorated\".to_string();") + || generated.contains("match {\n let __incan_static_arg_0 = \"decorated\".into();"), + "static receiver match scrutinee should materialize args inside an expression block:\n{}", + generated, ); Ok(()) } #[test] - fn e2e_yield_fixture_teardown_failure_fails_run() -> Result<(), Box> { + fn e2e_directory_run_preserves_per_file_inline_test_modules_issue676() -> Result<(), Box> { let dir = write_test_project( - "test_yield_fixture_teardown_failure.incn", + "incan.toml", + r#"[project] +name = "inline_directory_batch" +version = "0.1.0" +"#, + ); + let src_dir = dir.join("src"); + std::fs::create_dir_all(&src_dir)?; + std::fs::write( + src_dir.join("alpha.incn"), r#" -from std.testing import assert_eq, fixture +const ALPHA_OFFSET: int = 10 +static alpha_runs: int = 0 -@fixture -def resource() -> int: - yield 42 - assert_eq(1, 2) +model AlphaRecord: + value: int + label: str -def test_body_passes(resource: int) -> None: - assert_eq(resource, 42) +def alpha_value() -> int: + return 1 + +def alpha_record() -> AlphaRecord: + return AlphaRecord(value=alpha_value() + ALPHA_OFFSET, label="alpha") + + +module tests: + def test_alpha_value() -> None: + alpha_runs += 1 + record = alpha_record() + assert alpha_value() == 1 + assert record.value == 11 + assert record.label == "alpha" + assert alpha_runs == 1 "#, - ); + )?; + std::fs::write( + src_dir.join("beta.incn"), + r#" +const BETA_OFFSET: int = 20 +static beta_runs: int = 0 - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - !output.status.success(), - "expected teardown failure to fail the run.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains("test_body_passes FAILED") || stderr.contains("test_body_passes"), - "expected passing body with failing teardown to be reported as failed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - Ok(()) - } +model BetaRecord: + value: int + label: str - #[test] - fn e2e_yield_fixture_teardown_failures_are_aggregated() -> Result<(), Box> { - let dir = write_test_project( - "test_yield_fixture_teardown_aggregate.incn", +def beta_value() -> int: + return 2 + +def beta_record() -> BetaRecord: + return BetaRecord(value=beta_value() + BETA_OFFSET, label="beta") + + +module tests: + def test_beta_value() -> None: + beta_runs += 1 + record = beta_record() + assert beta_value() == 2 + assert record.value == 22 + assert record.label == "beta" + assert beta_runs == 1 +"#, + )?; + let functions_dir = src_dir.join("functions"); + std::fs::create_dir_all(&functions_dir)?; + std::fs::write( + functions_dir.join("columns.incn"), r#" -from std.testing import assert_eq, fixture +const COLUMN_OFFSET: int = 30 +static column_runs: int = 0 -@fixture -def parent() -> int: - yield 1 - assert_eq(1, 2, "parent teardown failed") +model Column: + value: int + label: str -@fixture -def child(parent: int) -> int: - yield parent + 1 - assert_eq(3, 4, "child teardown failed") +pub def col() -> int: + return 3 -def test_body_passes(child: int) -> None: - assert_eq(child, 2) +def column() -> Column: + return Column(value=col() + COLUMN_OFFSET, label="column") + + +module tests: + def test_col() -> None: + column_runs += 1 + item = column() + assert col() == 3 + assert item.value == 33 + assert item.label == "column" + assert column_runs == 1 +"#, + )?; + std::fs::write( + functions_dir.join("uses_columns.incn"), + r#" +from functions.columns import col + +const USES_COLUMN_OFFSET: int = 40 +static uses_column_runs: int = 0 + +model UsesColumn: + value: int + label: str + +def uses_col() -> int: + return col() + 1 + +def uses_column() -> UsesColumn: + return UsesColumn(value=uses_col() + USES_COLUMN_OFFSET, label="uses-column") + + +module tests: + def test_uses_col() -> None: + uses_column_runs += 1 + item = uses_column() + assert uses_col() == 4 + assert item.value == 44 + assert item.label == "uses-column" + assert uses_column_runs == 1 "#, + )?; + + let alpha = run_incan_test_path(&src_dir.join("alpha.incn")); + assert!( + alpha.status.success(), + "expected direct alpha inline test run to pass.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&alpha.stdout), + String::from_utf8_lossy(&alpha.stderr), + ); + let beta = run_incan_test_path(&src_dir.join("beta.incn")); + assert!( + beta.status.success(), + "expected direct beta inline test run to pass.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&beta.stdout), + String::from_utf8_lossy(&beta.stderr), + ); + let uses_columns = run_incan_test_path(&functions_dir.join("uses_columns.incn")); + assert!( + uses_columns.status.success(), + "expected direct imported inline test run to pass.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&uses_columns.stdout), + String::from_utf8_lossy(&uses_columns.stderr), + ); + + let directory = run_incan_test_path(&src_dir); + let stdout = String::from_utf8_lossy(&directory.stdout); + let stderr = String::from_utf8_lossy(&directory.stderr); + assert!( + directory.status.success(), + "expected directory inline test run to keep per-file parser context.\nstdout:\n{}\nstderr:\n{}", + stdout, + stderr, ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{stdout}\n{stderr}"); assert!( - !output.status.success(), - "expected aggregate teardown failures to fail the run.\nstdout:\n{}\nstderr:\n{}", + stdout.contains("alpha.incn::test_alpha_value") + && stdout.contains("beta.incn::test_beta_value") + && stdout.contains("columns.incn::test_col") + && stdout.contains("uses_columns.incn::test_uses_col"), + "expected every inline source file to run from directory discovery.\nstdout:\n{}", + stdout, + ); + assert!( + !stdout.contains("Only one `module tests:` block") && !stderr.contains("Only one `module tests:` block"), + "directory batching should not report duplicate inline modules across files.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); assert!( - combined.contains("fixture teardown failed") - && combined.contains("child teardown failed") - && combined.contains("parent teardown failed"), - "expected both teardown failures in aggregate output.\nstdout:\n{}\nstderr:\n{}", + !stderr.contains("the name `col` is defined multiple times"), + "directory batching should keep imported names inside their source module scope.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); + Ok(()) } #[test] - fn e2e_yield_fixture_teardown_captures_setup_locals() -> Result<(), Box> { + fn e2e_inline_module_parametrize_markers_strict_and_timeout() -> Result<(), Box> { let dir = write_test_project( - "test_yield_fixture_capture.incn", + "incan.toml", + r#"[project] +name = "inline_parametrize_markers" +version = "0.1.0" +"#, + ); + let src_dir = dir.join("src"); + std::fs::create_dir_all(&src_dir)?; + std::fs::write( + src_dir.join("math.incn"), r#" -from std.testing import assert_eq, fixture - -static observed: int = 0 +module tests: + from rust::std::thread import sleep + from rust::std::time import Duration + from std.testing import assert_eq, mark, param_case, parametrize, timeout, xfail -@fixture -def resource() -> int: - value: int = 41 - yield value + 1 - observed += value + const TEST_MARKERS: List[str] = ["smoke"] + const TEST_MARKS: List[str] = ["smoke"] -def test_body(resource: int) -> None: - assert_eq(resource, 42) + @parametrize("x, expected", [ + param_case((1, 3), marks=[xfail("known")], id="one-three"), + (2, 4), + ], ids=["ignored", "two-four"]) + def test_double(x: int, expected: int) -> None: + assert_eq(x * 2, expected) -def test_after_teardown() -> None: - assert_eq(observed, 41) + @mark("smoke") + @timeout("1ms") + def test_timeout_marker() -> None: + sleep(Duration.from_millis(100)) "#, - ); + )?; - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let listed = run_incan_test_with_args(&dir, &["--list", "-m", "smoke", "--strict-markers"]); + let listed_stdout = String::from_utf8_lossy(&listed.stdout); + let listed_stderr = String::from_utf8_lossy(&listed.stderr); assert!( - output.status.success(), - "expected yield teardown to capture setup locals.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + listed.status.success(), + "expected inline strict marker list to succeed.\nstdout:\n{}\nstderr:\n{}", + listed_stdout, + listed_stderr, ); - Ok(()) - } - - #[test] - fn e2e_module_yield_fixture_teardown_runs_at_module_boundary() -> Result<(), Box> { - let dir = write_test_project( - "test_module_yield_fixture.incn", - r#" -from std.testing import assert_eq, fixture - -static calls: int = 0 - -@fixture(scope="module") -def shared() -> int: - yield 10 - assert_eq(calls, 2) - -def test_first(shared: int) -> None: - calls += 1 - assert_eq(shared, 10) + assert!(listed_stdout.contains("src/math.incn::test_double[one-three]")); + assert!(listed_stdout.contains("src/math.incn::test_double[two-four]")); + assert!(listed_stdout.contains("src/math.incn::test_timeout_marker")); -def test_second(shared: int) -> None: - calls += 1 - assert_eq(shared, 10) -"#, + let run = run_incan_test_with_args(&dir, &["-k", "test_double"]); + let run_stdout = String::from_utf8_lossy(&run.stdout); + let run_stderr = String::from_utf8_lossy(&run.stderr); + assert!( + run.status.success(), + "expected inline parametrized xfail/pass cases to succeed.\nstdout:\n{}\nstderr:\n{}", + run_stdout, + run_stderr, ); + assert!(run_stdout.contains("XFAIL") || run_stdout.contains("xfailed")); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let timeout = run_incan_test_with_args(&dir, &["-k", "test_timeout_marker"]); + let timeout_stdout = String::from_utf8_lossy(&timeout.stdout); + let timeout_stderr = String::from_utf8_lossy(&timeout.stderr); assert!( - output.status.success(), - "expected module yield teardown after all tests in the file.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, + !timeout.status.success(), + "expected inline timeout marker to fail the test.\nstdout:\n{}\nstderr:\n{}", + timeout_stdout, + timeout_stderr, ); + assert!(timeout_stdout.contains("timed out after")); Ok(()) } #[test] - fn e2e_session_fixture_reused_across_files_with_single_worker() -> Result<(), Box> { + fn e2e_fixture_lifetime_success_scenarios_share_one_project() -> Result<(), Box> { let dir = write_test_project( "incan.toml", r#"[project] -name = "session_fixture_reuse" +name = "fixture_lifetime_success_batch" version = "0.1.0" "#, ); @@ -9437,234 +9518,247 @@ def test_b(session_value: int) -> None: assert_eq(session_value, 1) "#, )?; + std::fs::write( + tests_dir.join("test_fixture_lifetimes.incn"), + r#" +from std.async import sleep_ms +from std.testing import assert_eq, fixture, parametrize - let output = run_incan_test_with_args(&dir, &["--jobs", "1"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected session fixture to be reused across files in one worker batch.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - Ok(()) - } +static module_scope_calls: int = 0 +static yield_observed: int = 0 +static module_yield_calls: int = 0 +static teardown_order: int = 0 +static async_order: int = 0 +static async_reverse_order: str = "" +static async_param_setups: int = 0 - #[test] - fn e2e_yield_fixture_teardown_runs_in_reverse_dependency_order() -> Result<(), Box> { - let dir = write_test_project( - "test_yield_teardown_order.incn", - r#" -from std.testing import assert_eq, fixture +@fixture(scope="module") +def once() -> int: + module_scope_calls += 1 + return module_scope_calls + +def test_module_scope_first(once: int) -> None: + assert_eq(once, 1) + +def test_module_scope_second(once: int) -> None: + assert_eq(once, 1) -static order: int = 0 +@fixture +def captured_resource() -> int: + value: int = 41 + yield value + 1 + yield_observed += value + +def test_yield_capture_body(captured_resource: int) -> None: + assert_eq(captured_resource, 42) + +def test_yield_capture_after_teardown() -> None: + assert_eq(yield_observed, 41) + +@fixture(scope="module") +def module_shared() -> int: + yield 10 + assert_eq(module_yield_calls, 2) + +def test_module_yield_first(module_shared: int) -> None: + module_yield_calls += 1 + assert_eq(module_shared, 10) + +def test_module_yield_second(module_shared: int) -> None: + module_yield_calls += 1 + assert_eq(module_shared, 10) @fixture def outer() -> int: yield 1 - assert_eq(order, 1) - order += 1 + assert_eq(teardown_order, 1) + teardown_order += 1 @fixture def inner(outer: int) -> int: yield outer + 1 - assert_eq(order, 0) - order += 1 + assert_eq(teardown_order, 0) + teardown_order += 1 -def test_body(inner: int) -> None: +def test_reverse_teardown_body(inner: int) -> None: assert_eq(inner, 2) -def test_after() -> None: - assert_eq(order, 2) -"#, - ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected dependent fixtures to tear down in reverse dependency order.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - Ok(()) - } - - #[test] - fn e2e_async_yield_fixture_setup_and_teardown_are_awaited() -> Result<(), Box> { - let dir = write_test_project( - "test_async_yield_fixture.incn", - r#" -from std.async import sleep_ms -from std.testing import assert_eq, fixture - -static order: int = 0 +def test_reverse_teardown_after() -> None: + assert_eq(teardown_order, 2) @fixture def seed() -> int: - order += 1 + async_order += 1 return 40 @fixture async def resource(seed: int) -> int: await sleep_ms(1) - order += 1 + async_order += 1 yield seed + 2 await sleep_ms(1) - order += 10 + async_order += 10 def test_1_uses_async_fixture(resource: int) -> None: assert_eq(resource, 42) - assert_eq(order, 2) + assert_eq(async_order, 2) def test_2_observes_async_teardown() -> None: - assert_eq(order, 12) + assert_eq(async_order, 12) + +@fixture +async def parent() -> int: + async_reverse_order += "setup-parent;" + await sleep_ms(1) + yield 1 + await sleep_ms(1) + async_reverse_order += "teardown-parent;" + +@fixture +async def child(parent: int) -> int: + async_reverse_order += "setup-child;" + await sleep_ms(1) + yield parent + 1 + await sleep_ms(1) + async_reverse_order += "teardown-child;" + +def test_1_uses_child(child: int) -> None: + assert_eq(child, 2) + assert_eq(async_reverse_order, "setup-parent;setup-child;") + +def test_2_observes_reverse_teardown() -> None: + assert_eq(async_reverse_order, "setup-parent;setup-child;teardown-child;teardown-parent;") + +@fixture +async def base() -> int: + async_param_setups += 1 + await sleep_ms(1) + yield 10 + +@parametrize("value", [1, 2]) +async def test_param_async_fixture(value: int, base: int) -> None: + await sleep_ms(1) + assert_eq(base, 10) + assert_eq(value > 0, true) + +def test_after_param_cases() -> None: + assert_eq(async_param_setups, 2) "#, - ); + )?; - let output = run_incan_test(&dir); + let output = run_incan_test_with_args(&dir, &["--jobs", "1"]); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); assert!( output.status.success(), - "expected async fixture setup and teardown to be awaited.\nstdout:\n{}\nstderr:\n{}", + "expected fixture lifetime success batch to pass.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); + assert!(stdout.contains("test_module_scope_first") && stdout.contains("test_module_scope_second")); + assert!(stdout.contains("test_param_async_fixture[1]") && stdout.contains("test_param_async_fixture[2]")); Ok(()) } #[test] - fn e2e_async_yield_fixture_teardown_runs_after_failure() -> Result<(), Box> { + fn e2e_fixture_teardown_failure_scenarios_share_one_project() -> Result<(), Box> { let dir = write_test_project( - "test_async_yield_fixture_failure.incn", + "test_yield_fixture_teardown.incn", r#" -from std.async import sleep_ms from std.testing import assert_eq, fixture static calls: int = 0 @fixture -async def resource() -> int: +def resource() -> int: calls += 1 - await sleep_ms(1) yield calls - await sleep_ms(1) calls += 10 def test_1_fails(resource: int) -> None: assert_eq(resource, 99) -def test_2_observes_async_teardown() -> None: +def test_2_observes_teardown() -> None: assert_eq(calls, 11) "#, ); + std::fs::write( + dir.join("test_yield_fixture_teardown_failure.incn"), + r#" +from std.testing import assert_eq, fixture - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - !output.status.success(), - "expected the intentionally failing async-fixture test to fail the run.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - assert!( - stdout.contains("test_2_observes_async_teardown PASSED"), - "expected async teardown to run before the following test observed shared state.\nstdout:\n{}", - stdout, - ); - Ok(()) - } +@fixture +def resource() -> int: + yield 42 + assert_eq(1, 2) - #[test] - fn e2e_async_yield_fixture_teardown_runs_in_reverse_dependency_order() -> Result<(), Box> { - let dir = write_test_project( - "test_async_yield_teardown_order.incn", +def test_body_passes(resource: int) -> None: + assert_eq(resource, 42) +"#, + )?; + std::fs::write( + dir.join("test_yield_fixture_teardown_aggregate.incn"), r#" -from std.async import sleep_ms from std.testing import assert_eq, fixture -static order: str = "" - @fixture -async def parent() -> int: - order += "setup-parent;" - await sleep_ms(1) +def parent() -> int: yield 1 - await sleep_ms(1) - order += "teardown-parent;" + assert_eq(1, 2, "parent teardown failed") @fixture -async def child(parent: int) -> int: - order += "setup-child;" - await sleep_ms(1) +def child(parent: int) -> int: yield parent + 1 - await sleep_ms(1) - order += "teardown-child;" - -def test_1_uses_child(child: int) -> None: - assert_eq(child, 2) - assert_eq(order, "setup-parent;setup-child;") - -def test_2_observes_reverse_teardown() -> None: - assert_eq(order, "setup-parent;setup-child;teardown-child;teardown-parent;") -"#, - ); - - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - output.status.success(), - "expected async yield teardowns to run in reverse dependency order.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - Ok(()) - } + assert_eq(3, 4, "child teardown failed") - #[test] - fn e2e_async_fixture_composes_with_parametrize_before_resolution() -> Result<(), Box> { - let dir = write_test_project( - "test_async_param_fixture.incn", +def test_body_passes(child: int) -> None: + assert_eq(child, 2) +"#, + )?; + std::fs::write( + dir.join("test_async_yield_fixture_failure.incn"), r#" from std.async import sleep_ms -from std.testing import assert_eq, fixture, parametrize +from std.testing import assert_eq, fixture -static setups: int = 0 +static calls: int = 0 @fixture -async def base() -> int: - setups += 1 +async def resource() -> int: + calls += 1 await sleep_ms(1) - yield 10 - -@parametrize("value", [1, 2]) -async def test_param_async_fixture(value: int, base: int) -> None: + yield calls await sleep_ms(1) - assert_eq(base, 10) - assert_eq(value > 0, true) + calls += 10 -def test_after_param_cases() -> None: - assert_eq(setups, 2) +def test_1_fails(resource: int) -> None: + assert_eq(resource, 99) + +def test_2_observes_async_teardown() -> None: + assert_eq(calls, 11) "#, - ); + )?; let output = run_incan_test(&dir); let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); + let combined = format!("{stdout}\n{stderr}"); assert!( - output.status.success(), - "expected parametrized async tests to resolve async fixtures per expanded case.\nstdout:\n{}\nstderr:\n{}", + !output.status.success(), + "expected fixture teardown failure batch to fail.\nstdout:\n{}\nstderr:\n{}", stdout, stderr, ); assert!( - stdout.contains("test_param_async_fixture[1]") && stdout.contains("test_param_async_fixture[2]"), - "expected both parametrized async cases in reporter output.\nstdout:\n{}", + combined.contains("test_2_observes_teardown PASSED") + && combined.contains("test_2_observes_async_teardown PASSED") + && combined.contains("test_body_passes") + && combined.contains("fixture teardown failed") + && combined.contains("child teardown failed") + && combined.contains("parent teardown failed"), + "expected teardown diagnostics and observer tests in failure batch.\nstdout:\n{}\nstderr:\n{}", stdout, + stderr, ); Ok(()) } @@ -9760,202 +9854,90 @@ module tests: } #[test] - fn e2e_assert_failure_message_is_reported() { + fn e2e_failure_skip_and_assert_reporting_share_one_project() { let dir = write_test_project( - "test_assert_message.incn", + "test_failure_skip_and_assert_reporting.incn", r#" +from std.testing import assert_eq, skip + def test_message() -> None: assert False, "custom boom" + +def test_eq_message() -> None: + assert 1 == 2, "math broke" + +def test_wrong() -> None: + assert_eq(1 + 1, 99) + +@skip("not implemented yet") +def test_todo() -> None: + pass "#, ); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{stdout}\n{stderr}"); + let message = run_incan_test_with_args(&dir, &["-k", "test_message"]); + let message_stdout = String::from_utf8_lossy(&message.stdout); + let message_stderr = String::from_utf8_lossy(&message.stderr); + let message_combined = format!("{message_stdout}\n{message_stderr}"); assert!( - !output.status.success(), + !message.status.success(), "expected assertion failure test to fail.\n{}", - combined, + message_combined, ); assert!( - combined.contains("AssertionError: custom boom"), + message_combined.contains("AssertionError: custom boom"), "expected custom assertion message in output.\n{}", - combined, - ); - } - - #[test] - fn e2e_assert_eq_failure_reports_kind_and_message() { - let dir = write_test_project( - "test_assert_eq_message.incn", - r#" -def test_eq_message() -> None: - assert 1 == 2, "math broke" -"#, + message_combined, ); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{stdout}\n{stderr}"); + let eq = run_incan_test_with_args(&dir, &["-k", "test_eq_message"]); + let eq_stdout = String::from_utf8_lossy(&eq.stdout); + let eq_stderr = String::from_utf8_lossy(&eq.stderr); + let eq_combined = format!("{eq_stdout}\n{eq_stderr}"); assert!( - !output.status.success(), + !eq.status.success(), "expected assertion failure test to fail.\n{}", - combined, + eq_combined, ); assert!( - combined.contains("AssertionError: math broke"), + eq_combined.contains("AssertionError: math broke"), "expected custom equality assertion message in output.\n{}", - combined, + eq_combined, ); assert!( - combined.contains("left != right"), + eq_combined.contains("left != right"), "expected equality failure kind in output.\n{}", - combined, - ); - } - - // ---- Failing test ---- - - #[test] - fn e2e_failing_test_reports_failure() { - let dir = write_test_project( - "test_bad.incn", - r#" -from std.testing import assert_eq - -def test_wrong() -> None: - assert_eq(1 + 1, 99) -"#, + eq_combined, ); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); + let wrong = run_incan_test_with_args(&dir, &["-k", "test_wrong"]); + let wrong_stdout = String::from_utf8_lossy(&wrong.stdout); assert!( - !output.status.success(), + !wrong.status.success(), "expected failing test to exit non-zero.\nstdout:\n{}", - stdout, + wrong_stdout, ); assert!( - stdout.contains("FAILED") || stdout.contains("failed"), + wrong_stdout.contains("FAILED") || wrong_stdout.contains("failed"), "expected FAILED in output.\nstdout:\n{}", - stdout, - ); - } - - // ---- Skip marker ---- - - #[test] - fn e2e_skip_marker_skips_test() { - let dir = write_test_project( - "test_skip.incn", - r#" -from std.testing import skip - -@skip("not implemented yet") -def test_todo() -> None: - pass -"#, + wrong_stdout, ); - let output = run_incan_test(&dir); - let stdout = String::from_utf8_lossy(&output.stdout); + let skip = run_incan_test_with_args(&dir, &["-k", "test_todo"]); + let skip_stdout = String::from_utf8_lossy(&skip.stdout); assert!( - output.status.success(), + skip.status.success(), "expected skipped test to succeed overall.\nstdout:\n{}", - stdout, + skip_stdout, ); assert!( - stdout.contains("SKIPPED") || stdout.contains("skipped"), + skip_stdout.contains("SKIPPED") || skip_stdout.contains("skipped"), "expected SKIPPED in output.\nstdout:\n{}", - stdout, - ); - } - - // ---- Parametrize expansion ---- - - #[test] - fn e2e_parametrize_expands_and_runs_all_cases() { - let dir = write_test_project( - "test_param.incn", - r#" -from std.testing import parametrize, assert_eq - -@parametrize("a, b, expected", [(1, 2, 3), (10, 20, 30), (0, 0, 0)]) -def test_add(a: int, b: int, expected: int) -> None: - assert_eq(a + b, expected) -"#, - ); - - let output = run_incan_test_with_args(&dir, &["--verbose"]); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - - assert!( - output.status.success(), - "expected parametrized test to succeed.\nstdout:\n{}\nstderr:\n{}", - stdout, - stderr, - ); - - // All three parametrized variants should appear in the output. - assert!( - stdout.contains("test_add[1-2-3]"), - "expected test_add[1-2-3] in output.\nstdout:\n{}", - stdout, - ); - assert!( - stdout.contains("test_add[10-20-30]"), - "expected test_add[10-20-30] in output.\nstdout:\n{}", - stdout, - ); - assert!( - stdout.contains("test_add[0-0-0]"), - "expected test_add[0-0-0] in output.\nstdout:\n{}", - stdout, - ); - - // Should report 3 passed - assert!( - stdout.contains("3 passed"), - "expected '3 passed' in output.\nstdout:\n{}", - stdout, - ); - } - - // ---- Parametrize with a failing case ---- - - #[test] - fn e2e_parametrize_reports_failing_case() { - let dir = write_test_project( - "test_param_fail.incn", - r#" -from std.testing import parametrize, assert_eq - -@parametrize("x, expected", [(2, 4), (3, 7)]) -def test_double(x: int, expected: int) -> None: - assert_eq(x * 2, expected) -"#, - ); - - let output = run_incan_test_with_args(&dir, &["--verbose"]); - let stdout = String::from_utf8_lossy(&output.stdout); - - // 2*2==4 passes, 3*2==6!=7 fails - assert!( - !output.status.success(), - "expected one failing case to make the run fail.\nstdout:\n{}", - stdout, - ); - assert!( - stdout.contains("1 passed") && stdout.contains("1 failed"), - "expected '1 passed' and '1 failed'.\nstdout:\n{}", - stdout, + skip_stdout, ); } } @@ -10146,7 +10128,7 @@ def main() -> None: println(str(from_classmethod.value)) println(str(from_staticmethod.value)) "#; - let output = std::process::Command::new(super::incan_debug_binary()) + let output = super::incan_command() .args(["run", "-c", source]) .env("CARGO_NET_OFFLINE", "true") .output()?; @@ -10338,10 +10320,6 @@ mod rfc031_pub_import_integration_tests { use sha2::{Digest, Sha256}; use std::path::PathBuf; - fn incan_bin_path() -> std::path::PathBuf { - super::incan_debug_binary() - } - fn write_project_files( root: &Path, manifest_content: &str, @@ -10355,11 +10333,11 @@ mod rfc031_pub_import_integration_tests { } fn run_check(main_path: &Path) -> Result> { - Ok(Command::new(incan_bin_path()).arg("--check").arg(main_path).output()?) + Ok(super::incan_command().arg("--check").arg(main_path).output()?) } fn run_build(main_path: &Path, out_dir: &Path) -> Result> { - Ok(Command::new(incan_bin_path()) + Ok(super::incan_command() .args([ "build", main_path.to_string_lossy().as_ref(), @@ -10370,19 +10348,26 @@ mod rfc031_pub_import_integration_tests { } fn run_lock(entry_path: &Path) -> Result> { - Ok(Command::new(incan_bin_path()) + Ok(super::incan_command() .args(["lock", entry_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?) } fn run_test(target: &Path) -> Result> { - Ok(Command::new(incan_bin_path()) + Ok(super::incan_command() .args(["test", target.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") + .env("INCAN_TEST_SHARED_TARGET_DIR", shared_test_runner_target_dir()) .output()?) } + fn shared_test_runner_target_dir() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("incan_e2e_shared_target") + } + fn test_runner_batch_manifest_path(file_path: &Path) -> PathBuf { let canonical = std::fs::canonicalize(file_path).unwrap_or_else(|_| file_path.to_path_buf()); let mut hasher = Sha256::new(); @@ -10396,7 +10381,7 @@ mod rfc031_pub_import_integration_tests { } fn run_build_lib(project_root: &Path) -> Result> { - Ok(Command::new(incan_bin_path()) + Ok(super::incan_command() .args(["build", "--lib"]) .current_dir(project_root) .env("CARGO_NET_OFFLINE", "true") @@ -10404,175 +10389,131 @@ mod rfc031_pub_import_integration_tests { } #[test] - fn explicit_serialize_trait_adoption_runs_with_default_to_json() -> Result<(), Box> { + fn build_keeps_return_context_string_literal_union_arg_as_union_value() -> Result<(), Box> { let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"serialize_trait_default\"\n", - "from std.serde.json import Serialize\n\nmodel Payload with Serialize:\n value: int\n\ndef main() -> None:\n println(Payload(value=1).to_json())\n", + let project_root = tmp.path().join("return_context_union_arg"); + std::fs::create_dir_all(project_root.join("src"))?; + std::fs::write( + project_root.join("incan.toml"), + "[project]\nname = \"return_context_union_arg\"\nversion = \"0.1.0\"\n", )?; + std::fs::write( + project_root.join("src/projection_builders.incn"), + r#"pub model ColumnRefExpr: + column_name: str - let output = Command::new(incan_bin_path()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected explicit Serialize adoption to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); +pub model StringLiteralExpr: + value: str - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - stdout.contains("{\"value\":1}"), - "expected JSON output from default Serialize trait implementation, got:\n{}", - stdout - ); - Ok(()) - } +pub model FloatLiteralExpr: + value: float - #[test] - fn generated_runtime_helpers_run_for_pop_min_max_and_to_json() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"generated_runtime_helpers\"\nversion = \"0.3.0-dev.1\"\n", - "from std.serde.json import Serialize\n\nmodel Payload with Serialize:\n value: int\n\ndef main() -> None:\n mut xs = [3, 1, 4]\n println(xs.pop())\n println(min(xs))\n println(max(xs))\n println(Payload(value=2).to_json())\n", - )?; +pub model EqExpr: + arguments: list[ColumnExpr] - let output = Command::new(incan_bin_path()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; +pub type ColumnExpr = Union[ColumnRefExpr, StringLiteralExpr, FloatLiteralExpr, EqExpr] - assert!( - output.status.success(), - "expected generated runtime helper path project to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); +pub def col(name: str) -> ColumnExpr: + return ColumnRefExpr(column_name=name) - let stdout = String::from_utf8_lossy(&output.stdout); - let lines: Vec<&str> = stdout.lines().collect(); - assert_eq!( - lines.first().copied(), - Some("4"), - "expected xs.pop() output first, got:\n{stdout}" - ); - assert_eq!( - lines.get(1).copied(), - Some("1"), - "expected min(xs) after pop, got:\n{stdout}" - ); - assert_eq!( - lines.get(2).copied(), - Some("3"), - "expected max(xs) after pop, got:\n{stdout}" - ); - assert_eq!( - lines.get(3).copied(), - Some("{\"value\":2}"), - "expected Payload.to_json() output, got:\n{stdout}" - ); - Ok(()) - } +pub def str_expr(value: str) -> ColumnExpr: + return StringLiteralExpr(value=value) - #[test] - fn std_json_deserialize_from_json_runs_through_incan_surface() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"std_json_deserialize_from_json\"\nversion = \"0.3.0-dev.1\"\n", - r#"from std.serde import json +pub def float_expr(value: float) -> ColumnExpr: + return FloatLiteralExpr(value=value) -@derive(json) -model Payload: - value: int - label: str +pub def lit(value: Union[int, float, str, bool]) -> ColumnExpr: + match value: + float(number) => return float_expr(number) + str(text) => return str_expr(text) + bool(flag) => return str_expr("bool") + int(number) => return str_expr("int") -def main() -> None: - match Payload.from_json('{"value":7,"label":"dogfood"}'): - case Ok(payload): - println(payload.to_json()) - case Err(err): - println(err) +pub def eq(left: ColumnExpr, right: ColumnExpr) -> ColumnExpr: + return EqExpr(arguments=[left, right]) "#, )?; + std::fs::write( + project_root.join("src/functions.incn"), + "from projection_builders import col as col_builder, eq as eq_builder, lit as lit_builder\n\npub col = alias col_builder\npub lit = alias lit_builder\npub eq = alias eq_builder\n", + )?; + std::fs::write( + project_root.join("src/dataset.incn"), + r#"from projection_builders import ColumnExpr - let output = Command::new(incan_bin_path()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected std JSON Deserialize.from_json to run successfully through Incan source.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - - let stdout = String::from_utf8_lossy(&output.stdout); - assert_eq!( - stdout.lines().next(), - Some("{\"value\":7,\"label\":\"dogfood\"}"), - "expected round-tripped JSON payload, got:\n{stdout}" - ); - Ok(()) - } +pub class LazyFrame[T with Clone]: + pub rows: list[T] - #[test] - fn direct_std_json_deserialize_derive_runs_through_incan_surface() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"direct_std_json_deserialize_derive\"\nversion = \"0.3.0-dev.1\"\n", - r#"from std.serde.json import Deserialize + def filter(self, predicate: ColumnExpr) -> Self: + return self +"#, + )?; + let main_path = project_root.join("src/main.incn"); + std::fs::write( + &main_path, + r#"from dataset import LazyFrame +from functions import col, eq, lit -@derive(Deserialize) -model Payload: - value: int +model OrderLine: + status: str + discount: float + +def repro(lines: LazyFrame[OrderLine]) -> LazyFrame[OrderLine]: + return lines.filter(eq(col("status"), lit("open"))).filter(eq(col("discount"), lit(0.9))) def main() -> None: - match Payload.from_json('{"value":7}'): - case Ok(payload): - println(f"{payload.value}") - case Err(err): - println(err) + lines: LazyFrame[OrderLine] = LazyFrame[OrderLine](rows=[]) + _ = repro(lines) + println("done") "#, )?; - let output = Command::new(incan_bin_path()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - + let out_dir = project_root.join("out"); + let output = run_build(&main_path, &out_dir)?; assert!( output.status.success(), - "expected directly imported Deserialize.from_json to run successfully through Incan source.\nstdout:\n{}\nstderr:\n{}", + "expected union literal regression build to succeed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); - - let stdout = String::from_utf8_lossy(&output.stdout); - assert_eq!(stdout.lines().next(), Some("7")); + let generated_main = std::fs::read_to_string(out_dir.join("src/main.rs"))?; + let normalized: String = generated_main.chars().filter(|c| !c.is_whitespace()).collect(); + assert!( + normalized.contains("lit(crate::__IncanUnion43fbd19e99c1db05::V0(\"open\".to_string()))"), + "expected string literal to be wrapped directly as the union string arm, got:\n{generated_main}" + ); + assert!( + !normalized.contains("V0(\"open\".to_string()).to_string()"), + "union wrapper must not receive a post-wrapper string coercion, got:\n{generated_main}" + ); Ok(()) } #[test] - fn std_json_value_model_field_roundtrips_and_indexes() -> Result<(), Box> { + fn std_json_and_generated_runtime_surfaces_share_one_generated_run() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let main_path = write_project_files( tmp.path(), - "[project]\nname = \"json_value_model_field_roundtrip\"\nversion = \"0.3.0-dev.1\"\n", + "[project]\nname = \"std_json_runtime_surface_batch\"\nversion = \"0.3.0-dev.1\"\n", r#"from std.serde import json +from std.serde.json import Deserialize, Serialize from std.json import JsonValue +model SerializePayload with Serialize: + value: int + +model HelperPayload with Serialize: + value: int + +@derive(json) +model JsonPayload: + value: int + label: str + +@derive(Deserialize) +model DirectPayload: + value: int + @derive(json) model Envelope: status: int @@ -10584,7 +10525,33 @@ model Probe: first: Option[JsonValue] missing: Option[JsonValue] -def main() -> None: +const NUMBERS: FrozenList[float] = [3.0, 1.5, 4.25] + +def run_explicit_serialize_trait() -> None: + println(SerializePayload(value=1).to_json()) + +def run_generated_runtime_helpers() -> None: + mut xs = [3, 1, 4] + println(xs.pop()) + println(min(xs)) + println(max(xs)) + println(HelperPayload(value=2).to_json()) + +def run_std_json_deserialize() -> None: + match JsonPayload.from_json('{"value":7,"label":"dogfood"}'): + case Ok(payload): + println(payload.to_json()) + case Err(err): + println(err) + +def run_direct_deserialize_derive() -> None: + match DirectPayload.from_json('{"value":7}'): + case Ok(payload): + println(f"{payload.value}") + case Err(err): + println(err) + +def run_json_value_model_field_roundtrip() -> None: match Envelope.from_json('{"status":200,"data":{"name":"Ada","items":[1,2]}}'): case Ok(envelope): match envelope.data["items"]: @@ -10595,40 +10562,8 @@ def main() -> None: println("missing items") case Err(err): println(err) -"#, - )?; - - let output = Command::new(incan_bin_path()) - .arg("run") - .arg(&main_path) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - - assert!( - output.status.success(), - "expected JsonValue model-field round trip to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - - let stdout = String::from_utf8_lossy(&output.stdout); - assert_eq!( - stdout.lines().next(), - Some("{\"name\":\"Ada\",\"first\":1,\"missing\":null}"), - "expected checked JsonValue indexing to produce optional fields, got:\n{stdout}" - ); - Ok(()) - } - - #[test] - fn std_json_value_broad_surface_runs() -> Result<(), Box> { - let output = Command::new(incan_debug_binary()) - .args([ - "run", - "-c", - r#"from std.json import JsonValue -def main() -> None: +def run_std_json_value_broad_surface() -> None: match JsonValue.parse('{"items":[1,2],"name":"Ada","n":null}'): case Ok(data): assert data.kind().as_str() == "object" @@ -10675,30 +10610,23 @@ def main() -> None: case Err(err): println(err.message()) assert false -"#, - ]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - output.status.success(), - "expected std.json broad surface smoke program to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - Ok(()) - } +def run_frozen_float_helpers() -> None: + println(min(NUMBERS)) + println(max(NUMBERS)) - #[test] - fn generated_runtime_helpers_support_frozen_float_list_min_max() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"generated_runtime_helpers_frozen_float\"\nversion = \"0.3.0-dev.1\"\n", - "const NUMBERS: FrozenList[float] = [3.0, 1.5, 4.25]\n\ndef main() -> None:\n println(min(NUMBERS))\n println(max(NUMBERS))\n", +def main() -> None: + run_explicit_serialize_trait() + run_generated_runtime_helpers() + run_std_json_deserialize() + run_direct_deserialize_derive() + run_json_value_model_field_roundtrip() + run_std_json_value_broad_surface() + run_frozen_float_helpers() +"#, )?; - let output = Command::new(incan_bin_path()) + let output = super::incan_command() .arg("run") .arg(&main_path) .env("CARGO_NET_OFFLINE", "true") @@ -10706,22 +10634,27 @@ def main() -> None: assert!( output.status.success(), - "expected frozen-list min/max helper path project to run successfully.\nstdout:\n{}\nstderr:\n{}", + "expected std/json and generated runtime surface batch to run successfully.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); let stdout = String::from_utf8_lossy(&output.stdout); - let lines: Vec<&str> = stdout.lines().collect(); assert_eq!( - lines.first().copied(), - Some("1.5"), - "expected min(NUMBERS) output first, got:\n{stdout}" - ); - assert_eq!( - lines.get(1).copied(), - Some("4.25"), - "expected max(NUMBERS) output second, got:\n{stdout}" + stdout.lines().collect::>(), + vec![ + "{\"value\":1}", + "4", + "1", + "3", + "{\"value\":2}", + "{\"value\":7,\"label\":\"dogfood\"}", + "7", + "{\"name\":\"Ada\",\"first\":1,\"missing\":null}", + "1.5", + "4.25", + ], + "expected std/json and generated runtime surface transcript, got:\n{stdout}" ); Ok(()) } @@ -10871,535 +10804,15 @@ pub def display[T](data: DataSet[T]) -> None: std::fs::create_dir_all(crate_root.join("src"))?; std::fs::write( crate_root.join("Cargo.toml"), - format!( - "[package]\nname = \"{package_name}\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[dependencies]\nincan_vocab = {{ path = \"{}\" }}\n\n[lib]\npath = \"src/lib.rs\"\n", - Path::new(env!("CARGO_MANIFEST_DIR")) - .join("crates") - .join("incan_vocab") - .display() - ), - )?; - std::fs::write(crate_root.join("src/lib.rs"), lib_source)?; - Ok(()) - } - - fn write_nested_wasm_vocab_companion_crate( - project_root: &Path, - relative_path: &str, - package_name: &str, - ) -> Result<(), Box> { - let crate_root = project_root.join(relative_path); - std::fs::create_dir_all(crate_root.join("src"))?; - std::fs::write( - crate_root.join("Cargo.toml"), - format!( - "[package]\nname = \"{package_name}\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[lib]\npath = \"src/lib.rs\"\ncrate-type = [\"rlib\", \"cdylib\"]\n\n[dependencies]\nincan_vocab = {{ path = \"{}\" }}\n\n[workspace]\n", - Path::new(env!("CARGO_MANIFEST_DIR")) - .join("crates") - .join("incan_vocab") - .display() - ), - )?; - std::fs::write( - crate_root.join("src/lib.rs"), - r#"use incan_vocab::{ - DesugarError, DesugarOutput, HelperBinding, IncanExpr, IncanStatement, KeywordActivation, - KeywordPlacement, KeywordRegistration, KeywordSpec, KeywordSurfaceKind, LibraryManifest, - VocabBodyItem, VocabClause, VocabClauseBody, VocabDeclaration, VocabDesugarer, - VocabFieldSpec, VocabRegistration, VocabSyntaxNode, -}; - -#[derive(Default)] -pub struct NestedOutputDesugarer; - -pub fn library_vocab() -> VocabRegistration { - VocabRegistration::new() - .with_keyword_registration(KeywordRegistration { - activation: KeywordActivation::OnImport { - namespace: "nested.dsl".to_string(), - }, - keywords: vec![ - KeywordSpec::new("compose", KeywordSurfaceKind::BlockDeclaration), - context_keyword("action", &["compose"]), - context_keyword("layout", &["compose"]), - context_keyword("page", &["compose"]), - context_keyword("projection", &["compose"]), - context_keyword("region", &["layout", "page"]), - context_keyword("heading", &["region"]), - context_keyword("text", &["region"]), - context_keyword("interaction", &["page"]), - context_keyword("require", &["interaction"]), - ], - valid_decorators: Vec::new(), - }) - .with_library_manifest(LibraryManifest { - helper_bindings: vec![ - helper_binding("action"), - helper_binding("layout"), - helper_binding("page_with_interactions"), - helper_binding("projection"), - helper_binding("region"), - helper_binding("heading"), - helper_binding("text"), - helper_binding("interaction"), - helper_binding("required_input"), - helper_binding("surface_with_governance"), - ], - ..LibraryManifest::default() - }) - .with_desugarer(NestedOutputDesugarer) -} - -impl VocabDesugarer for NestedOutputDesugarer { - fn desugar(&self, node: &VocabSyntaxNode) -> Result { - match node { - VocabSyntaxNode::Declaration(declaration) if declaration.keyword == "compose" => Ok( - DesugarOutput::Statements(vec![complex_artifact_let_statement(declaration)?]), - ), - VocabSyntaxNode::Declaration(_) => Err(DesugarError::new( - "nested output desugarer expected a compose declaration", - )), - _ => Err(DesugarError::new( - "nested output desugarer expected a declaration node", - )), - } - } -} - -fn helper_binding(name: &str) -> HelperBinding { - HelperBinding { - key: name.to_string(), - exported_name: name.to_string(), - } -} - -fn context_keyword(name: &str, parents: &[&str]) -> KeywordSpec { - KeywordSpec::new(name, KeywordSurfaceKind::BlockContextKeyword) - .with_placement(KeywordPlacement::in_block(parents.iter().copied())) -} - -fn complex_artifact_let_statement(declaration: &VocabDeclaration) -> Result { - Ok(IncanStatement::Let { - name: "nested_artifact".to_string(), - mutable: false, - value: complex_artifact_call(declaration)?, - }) -} - -fn complex_artifact_call(declaration: &VocabDeclaration) -> Result { - let name = declaration - .head - .name - .clone() - .ok_or_else(|| DesugarError::new("compose declarations require a name"))?; - let mut title = name.clone(); - let mut base = "/".to_string(); - let mut actions = Vec::new(); - let mut layouts = Vec::new(); - let mut pages = Vec::new(); - let mut projections = Vec::new(); - - for item in &declaration.body { - match item { - VocabBodyItem::Statement(statement) => apply_surface_statement(&mut title, &mut base, statement)?, - VocabBodyItem::Clause(clause) => match clause.keyword.as_str() { - "action" => actions.push(desugar_action(clause)?), - "layout" => layouts.push(desugar_layout(clause)?), - "page" => pages.push(desugar_page(clause)?), - "projection" => projections.push(desugar_projection(clause)?), - other => return Err(DesugarError::new(format!("unsupported compose clause `{other}`"))), - }, - VocabBodyItem::Declaration(declaration) => { - return Err(DesugarError::new(format!( - "unsupported nested declaration `{}`", - declaration.keyword - ))); - } - _ => return Err(DesugarError::new("unsupported compose body item")), - } - } - - Ok(call( - "surface_with_governance", - vec![ - string(&name), - string(&title), - string(&base), - list(actions), - list(layouts), - list(pages), - list(projections), - ], - )) -} - -fn apply_surface_statement(title: &mut String, base: &mut String, statement: &IncanStatement) -> Result<(), DesugarError> { - match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } => match name.as_str() { - "title" => *title = string_value(value, "compose title")?, - "base" => *base = string_value(value, "compose base")?, - other => return Err(DesugarError::new(format!("unsupported compose assignment `{other}`"))), - }, - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("compose body only supports assignments and nested clauses")), - } - Ok(()) -} - -fn desugar_action(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "action")?; - let mut capability = name.clone(); - let mut required_evidence = String::new(); - match &clause.body { - VocabClauseBody::FieldSet(fields) => { - for field in fields { - apply_action_assignment( - &mut capability, - &mut required_evidence, - &field.name, - field_value(field, "action assignment")?, - )?; - } - } - _ => { - for item in clause_items(clause)? { - match item { - VocabBodyItem::Statement(statement) => match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } => { - apply_action_assignment(&mut capability, &mut required_evidence, name, value)?; - } - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("action body only supports assignments")), - }, - VocabBodyItem::Clause(other) => return Err(DesugarError::new(format!("unsupported `{}` block inside action", other.keyword))), - VocabBodyItem::Declaration(_) => return Err(DesugarError::new("action body only supports assignments")), - _ => return Err(DesugarError::new("action body only supports assignments")), - } - } - } - } - Ok(call("action", vec![string(&name), string(&capability), string(&required_evidence)])) -} - -fn apply_action_assignment( - capability: &mut String, - required_evidence: &mut String, - name: &str, - value: &IncanExpr, -) -> Result<(), DesugarError> { - match name { - "capability" => *capability = string_value(value, "action capability")?, - "requires" | "required_evidence" => *required_evidence = string_value(value, "action required evidence")?, - other => return Err(DesugarError::new(format!("unsupported action assignment `{other}`"))), - } - Ok(()) -} - -fn desugar_layout(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "layout")?; - let mut regions = Vec::new(); - for item in clause_items(clause)? { - match item { - VocabBodyItem::Clause(region_clause) if region_clause.keyword == "region" => { - regions.push(string(&required_head_name(region_clause, "layout region")?)); - } - VocabBodyItem::Statement(IncanStatement::Pass) => {} - VocabBodyItem::Clause(other) => return Err(DesugarError::new(format!("unsupported `{}` block inside layout", other.keyword))), - _ => return Err(DesugarError::new("layout body only supports region blocks")), - } - } - Ok(call("layout", vec![string(&name), list(regions)])) -} - -fn desugar_page(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "page")?; - let mut route = "/".to_string(); - let mut title = name.clone(); - let mut layout_name = "SimplePage".to_string(); - let mut regions = Vec::new(); - let mut interactions = Vec::new(); - for item in clause_items(clause)? { - match item { - VocabBodyItem::Statement(statement) => match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } => match name.as_str() { - "route" => route = string_value(value, "page route")?, - "title" => title = string_value(value, "page title")?, - "layout" => layout_name = string_or_name_value(value, "page layout")?, - other => return Err(DesugarError::new(format!("unsupported page assignment `{other}`"))), - }, - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("page body only supports assignments, regions, and interactions")), - }, - VocabBodyItem::Clause(region_clause) if region_clause.keyword == "region" => { - regions.push(desugar_region(region_clause)?); - } - VocabBodyItem::Clause(interaction_clause) if interaction_clause.keyword == "interaction" => { - interactions.push(desugar_interaction(interaction_clause)?); - } - VocabBodyItem::Clause(other) => return Err(DesugarError::new(format!("unsupported `{}` block inside page", other.keyword))), - VocabBodyItem::Declaration(_) => return Err(DesugarError::new("page body does not support nested declarations")), - _ => return Err(DesugarError::new("page body only supports assignments, regions, and interactions")), - } - } - Ok(call( - "page_with_interactions", - vec![ - string(&name), - string(&route), - string(&title), - string(&layout_name), - list(regions), - list(interactions), - ], - )) -} - -fn desugar_region(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "region")?; - let mut nodes = Vec::new(); - for item in clause_items(clause)? { - match item { - VocabBodyItem::Clause(node_clause) if node_clause.keyword == "heading" => { - nodes.push(call("heading", vec![string(&required_head_string(node_clause, "heading")?)])); - } - VocabBodyItem::Clause(node_clause) if node_clause.keyword == "text" => { - nodes.push(call("text", vec![string(&required_head_string(node_clause, "text")?)])); - } - VocabBodyItem::Statement(IncanStatement::Pass) => {} - VocabBodyItem::Clause(other) => return Err(DesugarError::new(format!("unsupported `{}` block inside region", other.keyword))), - _ => return Err(DesugarError::new("region body only supports heading and text blocks")), - } - } - Ok(call("region", vec![string(&name), list(nodes)])) -} - -fn desugar_interaction(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "interaction")?; - let mut action = name.clone(); - let mut constraints = Vec::new(); - match &clause.body { - VocabClauseBody::FieldSet(fields) => { - for field in fields { - if field.name == "action" { - action = string_or_name_value(field_value(field, "interaction action")?, "interaction action")?; - } else { - return Err(DesugarError::new(format!("unsupported interaction assignment `{}`", field.name))); - } - } - } - _ => { - for item in clause_items(clause)? { - match item { - VocabBodyItem::Statement(statement) => match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } => match name.as_str() { - "action" => action = string_or_name_value(value, "interaction action")?, - other => return Err(DesugarError::new(format!("unsupported interaction assignment `{other}`"))), - }, - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("interaction body only supports assignments and require blocks")), - }, - VocabBodyItem::Clause(require_clause) if require_clause.keyword == "require" => { - constraints.push(desugar_required_input(&name, require_clause)?); - } - VocabBodyItem::Clause(other) => return Err(DesugarError::new(format!("unsupported `{}` block inside interaction", other.keyword))), - VocabBodyItem::Declaration(_) => return Err(DesugarError::new("interaction body does not support nested declarations")), - _ => return Err(DesugarError::new("interaction body only supports assignments and require blocks")), - } - } - } - } - Ok(call("interaction", vec![string(&name), string(&action), list(constraints)])) -} - -fn desugar_required_input(interaction_name: &str, clause: &VocabClause) -> Result { - let mut field = required_input_field(clause)?; - let mut label = field.clone(); - let mut min_length = "1".to_string(); - let mut evidence_key = String::new(); - match &clause.body { - VocabClauseBody::FieldSet(fields) => { - for field_spec in fields { - apply_required_input_assignment( - &mut field, - &mut label, - &mut min_length, - &mut evidence_key, - &field_spec.name, - field_value(field_spec, "require input assignment")?, - )?; - } - } - _ => { - for item in clause_items(clause)? { - match item { - VocabBodyItem::Statement(statement) => match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } => { - apply_required_input_assignment( - &mut field, - &mut label, - &mut min_length, - &mut evidence_key, - name, - value, - )?; - } - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("require input body only supports assignments")), - }, - _ => return Err(DesugarError::new("require input body only supports assignments")), - } - } - } - } - Ok(call( - "required_input", - vec![ - string(interaction_name), - string(&field), - string(&label), - string(&min_length), - string(&evidence_key), - ], - )) -} - -fn apply_required_input_assignment( - field: &mut String, - label: &mut String, - min_length: &mut String, - evidence_key: &mut String, - name: &str, - value: &IncanExpr, -) -> Result<(), DesugarError> { - match name { - "field" => *field = string_or_name_value(value, "required input field")?, - "label" => *label = string_value(value, "required input label")?, - "min_length" => *min_length = int_or_string_value(value, "required input min_length")?, - "evidence" | "evidence_key" => *evidence_key = string_value(value, "required input evidence")?, - other => return Err(DesugarError::new(format!("unsupported require input assignment `{other}`"))), - } - Ok(()) -} - -fn desugar_projection(clause: &VocabClause) -> Result { - let name = required_head_name(clause, "projection")?; - let mut target = "static-web".to_string(); - match &clause.body { - VocabClauseBody::FieldSet(fields) => { - for field in fields { - if field.name == "target" { - target = string_value(field_value(field, "projection target")?, "projection target")?; - } else { - return Err(DesugarError::new(format!("unsupported projection assignment `{}`", field.name))); - } - } - } - _ => { - for item in clause_items(clause)? { - match item { - VocabBodyItem::Statement(statement) => match statement { - IncanStatement::Let { name, value, .. } | IncanStatement::Assign { target: name, value } if name == "target" => { - target = string_value(value, "projection target")?; - } - IncanStatement::Pass => {} - _ => return Err(DesugarError::new("projection body only supports target assignment")), - }, - _ => return Err(DesugarError::new("projection body only supports target assignment")), - } - } - } - } - Ok(call("projection", vec![string(&name), string(&target)])) -} - -fn clause_items(clause: &VocabClause) -> Result<&[VocabBodyItem], DesugarError> { - match &clause.body { - VocabClauseBody::Empty => Ok(&[]), - VocabClauseBody::Items(items) => Ok(items.as_slice()), - _ => Err(DesugarError::new(format!("unsupported `{}` body shape", clause.keyword))), - } -} - -fn required_head_name(clause: &VocabClause, label: &str) -> Result { - let Some(value) = clause.head.first() else { - return Err(DesugarError::new(format!("{label} requires a name"))); - }; - string_or_name_value(value, label) -} - -fn required_head_string(clause: &VocabClause, label: &str) -> Result { - let Some(value) = clause.head.first() else { - return Err(DesugarError::new(format!("{label} requires text"))); - }; - string_value(value, label) -} - -fn required_input_field(clause: &VocabClause) -> Result { - if !clause.compound_tokens.is_empty() && clause.compound_tokens[0] == "input" { - if let Some(value) = clause.head.first() { - return string_or_name_value(value, "require input field"); - } - return Ok(String::new()); - } - if !clause.head.is_empty() { - let first = string_or_name_value(&clause.head[0], "require input marker")?; - if first == "input" { - if clause.head.len() >= 2 { - return string_or_name_value(&clause.head[1], "require input field"); - } - return Ok(String::new()); - } - } - Err(DesugarError::new("required-input constraints must use `require input`")) -} - -fn string_value(expr: &IncanExpr, label: &str) -> Result { - match expr { - IncanExpr::Str(value) => Ok(value.clone()), - _ => Err(DesugarError::new(format!("{label} must be a string literal"))), - } -} - -fn string_or_name_value(expr: &IncanExpr, label: &str) -> Result { - match expr { - IncanExpr::Str(value) | IncanExpr::Name(value) => Ok(value.clone()), - _ => Err(DesugarError::new(format!("{label} must be a name or string literal"))), - } -} - -fn int_or_string_value(expr: &IncanExpr, label: &str) -> Result { - match expr { - IncanExpr::Int(value) => Ok(value.to_string()), - IncanExpr::Str(value) => Ok(value.clone()), - _ => Err(DesugarError::new(format!("{label} must be an integer or string literal"))), - } -} - -fn field_value<'a>(field: &'a VocabFieldSpec, label: &str) -> Result<&'a IncanExpr, DesugarError> { - field - .default_value - .as_ref() - .ok_or_else(|| DesugarError::new(format!("{label} `{}` requires a value", field.name))) -} - -fn call(helper: &str, args: Vec) -> IncanExpr { - IncanExpr::Call { - callee: Box::new(IncanExpr::Helper(helper.to_string())), - args, - } -} - -fn list(items: Vec) -> IncanExpr { - IncanExpr::List(items) -} - -fn string(value: &str) -> IncanExpr { - IncanExpr::Str(value.to_string()) -} - -incan_vocab::export_wasm_desugarer!(NestedOutputDesugarer); -"#, + format!( + "[package]\nname = \"{package_name}\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[dependencies]\nincan_vocab = {{ path = \"{}\" }}\n\n[lib]\npath = \"src/lib.rs\"\n", + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("crates") + .join("incan_vocab") + .display() + ), )?; + std::fs::write(crate_root.join("src/lib.rs"), lib_source)?; Ok(()) } @@ -11916,10 +11329,12 @@ incan_vocab::export_wasm_desugarer!(NestedOutputDesugarer); Ok(()) } - fn write_pub_library_with_assert_keyword( + fn write_pub_library_with_provider_requirements_and_assert_keyword( root: &Path, dependency_key: &str, manifest_name: &str, + required_dependencies: Vec, + required_stdlib_features: Vec<&str>, ) -> Result<(), Box> { let artifact_root = root.join("deps").join(dependency_key).join("target").join("lib"); std::fs::create_dir_all(artifact_root.join("src"))?; @@ -11940,7 +11355,14 @@ incan_vocab::export_wasm_desugarer!(NestedOutputDesugarer); valid_decorators: Vec::new(), }], dsl_surfaces: Vec::new(), - provider_manifest: incan_vocab::LibraryManifest::default(), + provider_manifest: incan_vocab::LibraryManifest { + required_dependencies, + required_stdlib_features: required_stdlib_features + .into_iter() + .map(std::string::ToString::to_string) + .collect(), + ..incan_vocab::LibraryManifest::default() + }, desugarer_artifact: None, }); manifest.write_to_path(&artifact_root.join(format!("{manifest_name}.incnlib")))?; @@ -12108,579 +11530,424 @@ incan_vocab::export_wasm_desugarer!(NestedOutputDesugarer); let output = run_check(&main_path)?; assert!( - !output.status.success(), - "expected check to fail for missing crate artifacts, stderr:\n{}", - String::from_utf8_lossy(&output.stderr) - ); - let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&output.stderr)); - assert!( - stderr.contains("Missing generated crate artifacts for `pub::mylib`"), - "expected missing-artifact diagnostic, got:\n{stderr}" - ); - Ok(()) - } - - #[test] - fn check_reports_pub_library_artifact_mismatch() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let dep_artifact_root = tmp.path().join("deps").join("widgets-lib").join("target").join("lib"); - std::fs::create_dir_all(&dep_artifact_root)?; - let mut manifest = LibraryManifest::new("widgets_core", "0.1.0"); - manifest.exports.models.push(ModelExport { - name: "Widget".to_string(), - type_params: Vec::new(), - traits: Vec::new(), - trait_adoptions: Vec::new(), - derives: Vec::new(), - fields: Vec::new(), - methods: Vec::new(), - }); - manifest.write_to_path(&dep_artifact_root.join("widgets_core.incnlib"))?; - write_minimal_library_crate(&dep_artifact_root, "different_package_name")?; - - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"app\"\n\n[dependencies]\nwidgets = { path = \"deps/widgets-lib\" }\n", - "from pub::widgets import Widget\n", - )?; - - let output = run_check(&main_path)?; - assert!( - !output.status.success(), - "expected check to fail for artifact mismatch, stderr:\n{}", - String::from_utf8_lossy(&output.stderr) - ); - let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&output.stderr)); - assert!( - stderr.contains("Generated crate metadata mismatch for `pub::widgets`"), - "expected artifact mismatch diagnostic, got:\n{stderr}" - ); - Ok(()) - } - - #[test] - fn build_lib_artifacts_and_consumer_alias_linkage() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let producer_root = tmp.path().join("widgets_core_project"); - std::fs::create_dir_all(producer_root.join("src"))?; - std::fs::write( - producer_root.join("incan.toml"), - "[project]\nname = \"widgets_core\"\nversion = \"0.1.0\"\n", - )?; - std::fs::write( - producer_root.join("src/widgets.incn"), - "pub model Widget:\n name: str\n\npub def make_widget(name: str) -> Widget:\n return Widget(name=name)\n", - )?; - std::fs::write( - producer_root.join("src/lib.incn"), - "pub from widgets import Widget, make_widget\n", - )?; - - let producer_build = run_build_lib(&producer_root)?; - assert!( - producer_build.status.success(), - "expected `build --lib` to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&producer_build.stdout), - String::from_utf8_lossy(&producer_build.stderr) - ); - let producer_artifact_root = producer_root.join("target").join("lib"); - assert!(producer_artifact_root.join("Cargo.toml").is_file()); - assert!(producer_artifact_root.join("src/lib.rs").is_file()); - assert!(producer_artifact_root.join("widgets_core.incnlib").is_file()); - - let consumer_root = tmp.path().join("consumer_app"); - std::fs::create_dir_all(consumer_root.join("src"))?; - std::fs::write( - consumer_root.join("incan.toml"), - "[project]\nname = \"consumer\"\n\n[dependencies]\nwidgets = { path = \"../widgets_core_project\" }\n", - )?; - let consumer_main = consumer_root.join("src/main.incn"); - std::fs::write( - &consumer_main, - "from pub::widgets import Widget as PublicWidget, make_widget\n\ndef main() -> None:\n w: PublicWidget = make_widget(\"ok\")\n print(w.name)\n", - )?; - - let out_dir = consumer_root.join("out"); - let consumer_build = run_build(&consumer_main, &out_dir)?; - assert!( - consumer_build.status.success(), - "expected consumer build to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&consumer_build.stdout), - String::from_utf8_lossy(&consumer_build.stderr) - ); - - let generated_toml = std::fs::read_to_string(out_dir.join("Cargo.toml"))?; - assert!( - generated_toml.contains("[dependencies.widgets]"), - "expected library alias dependency entry, got:\n{generated_toml}" - ); - assert!( - generated_toml.contains("package = \"widgets_core\""), - "expected package alias mapping in Cargo.toml, got:\n{generated_toml}" - ); - assert!( - generated_toml.contains("path = "), - "expected path dependency in Cargo.toml, got:\n{generated_toml}" - ); - - let generated_main_rs = std::fs::read_to_string(out_dir.join("src/main.rs"))?; - assert!( - generated_main_rs.contains("use widgets::Widget as PublicWidget;"), - "expected pub:: item alias import emission, got:\n{generated_main_rs}" - ); - assert!( - generated_main_rs.contains("use widgets::make_widget;"), - "expected pub:: item import emission, got:\n{generated_main_rs}" - ); - assert!( - !generated_main_rs.contains("pub use widgets::Widget as PublicWidget;"), - "private pub:: item alias import should not become a public Rust reexport, got:\n{generated_main_rs}" - ); - assert!( - !generated_main_rs.contains("pub use widgets::make_widget;"), - "private pub:: item import should not become a public Rust reexport, got:\n{generated_main_rs}" - ); - - Ok(()) - } - - #[test] - fn build_accepts_pub_from_reexport_in_src_submodule_facade() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("session_facade_project"); - std::fs::create_dir_all(project_root.join("src/session"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"session_facade\"\nversion = \"0.1.0\"\n", - )?; - std::fs::write( - project_root.join("src/session/types.incn"), - "pub class Session:\n pub id: int\n", - )?; - std::fs::write( - project_root.join("src/session/mod.incn"), - "pub from crate.session.types import Session\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "from session import Session\n\ndef main() -> None:\n s = Session(id=1)\n print(s.id)\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected `build` to accept src submodule facade re-export.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_imported_enum_loop_ownership() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("imported_enum_loop_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"imported_enum_loop\"\nversion = \"0.1.0\"\n", - )?; - std::fs::write( - project_root.join("src/rels.incn"), - "@derive(Clone)\npub enum ConformanceRel:\n Read\n Filter\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "from rels import ConformanceRel\n\ndef relation_kind_name_from_conformance(rel: ConformanceRel) -> str:\n match rel:\n ConformanceRel.Read =>\n return \"ReadRel\"\n _ =>\n return \"Other\"\n\ndef scenario_matches(required: list[ConformanceRel]) -> bool:\n for expected in required:\n if expected == ConformanceRel.Read:\n if relation_kind_name_from_conformance(expected) == \"ReadRel\":\n return true\n return false\n\ndef main() -> None:\n println(scenario_matches([ConformanceRel.Read]))\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected imported enum loop project to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_len_comparison_on_recursive_list_field() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("len_comparison_recursive_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"len_comparison_recursive\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "@derive(Clone)\npub enum ExprKind:\n Column\n Add\n\n@derive(Clone)\npub model Expr:\n pub kind: ExprKind\n pub column_name: str\n pub arguments: list[Expr]\n\npub def lower(expr: Expr) -> int:\n if expr.kind == ExprKind.Column:\n return 0\n if len(expr.arguments) < 2:\n return -1\n return 1\n\ndef main() -> None:\n println(lower(Expr(kind=ExprKind.Add, column_name=\"root\", arguments=[])))\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected recursive list-field len comparison project to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_loop_helper_shared_string_list() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("loop_helper_shared_string_list_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"loop_helper_shared_string_list\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "def match_index(xs: list[str], y: int) -> int:\n mut idx = 0\n while idx < len(xs):\n if len(xs[idx]) == y:\n return idx\n idx = idx + 1\n return -1\n\n\ -def helper_loop(xs: list[str], ys: list[int]) -> list[int]:\n mut out: list[int] = []\n for y in ys:\n out.append(match_index(xs, y))\n return out\n\n\ -def main() -> None:\n helper_loop([\"a\", \"bb\", \"ccc\"], [1, 2])\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected loop helper shared string-list project to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_dict_comp_reusing_noncopy_key() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("dict_comp_reuses_noncopy_key_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"dict_comp_reuses_noncopy_key\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "def lengths(names: list[str]) -> dict[str, int]:\n return {name: len(name) for name in names}\n\n\ -def main() -> None:\n values = lengths([\"alice\", \"bob\"])\n println(values[\"alice\"])\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected dict comprehension with reused non-Copy key to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_for_tuple_unpack_enumerate() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("for_tuple_unpack_enumerate_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"for_tuple_unpack_enumerate\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "model Binding:\n name: str\n output_index: int\n expr_index: int\n\n\ -def field_ref(index: int) -> int:\n return index\n\n\ -pub def bind(xs: list[str]) -> list[Binding]:\n mut out: list[Binding] = []\n for idx, name in enumerate(xs):\n out.append(Binding(name=name, output_index=idx, expr_index=field_ref(idx)))\n return out\n\n\ -def main() -> None:\n bind([\"a\", \"bb\"])\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected for-loop tuple unpacking with enumerate to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); - - Ok(()) - } - - #[test] - fn build_succeeds_for_list_comp_tuple_unpack_enumerate() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("list_comp_tuple_unpack_enumerate_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"list_comp_tuple_unpack_enumerate\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "model Binding:\n name: str\n output_index: int\n expr_index: int\n\n\ -def field_ref(index: int) -> int:\n return index\n\n\ -pub def bind(xs: list[str]) -> list[Binding]:\n return [Binding(name=name, output_index=idx, expr_index=field_ref(idx)) for idx, name in enumerate(xs)]\n\n\ -def main() -> None:\n bind([\"a\", \"bb\"])\n", - )?; - - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; + !output.status.success(), + "expected check to fail for missing crate artifacts, stderr:\n{}", + String::from_utf8_lossy(&output.stderr) + ); + let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&output.stderr)); assert!( - project_build.status.success(), - "expected list-comprehension tuple unpacking with enumerate to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) + stderr.contains("Missing generated crate artifacts for `pub::mylib`"), + "expected missing-artifact diagnostic, got:\n{stderr}" ); - Ok(()) } #[test] - fn build_succeeds_for_list_str_append_literal() -> Result<(), Box> { + fn check_reports_pub_library_artifact_mismatch() -> Result<(), Box> { let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("list_str_append_literal_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"list_str_append_literal\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "pub def columns(input_columns: list[str]) -> list[str]:\n mut columns: list[str] = []\n columns.append(input_columns[0])\n columns.append(\"count\")\n return columns\n\n\ -def main() -> None:\n columns([\"orders_total\"])\n", + let dep_artifact_root = tmp.path().join("deps").join("widgets-lib").join("target").join("lib"); + std::fs::create_dir_all(&dep_artifact_root)?; + let mut manifest = LibraryManifest::new("widgets_core", "0.1.0"); + manifest.exports.models.push(ModelExport { + name: "Widget".to_string(), + type_params: Vec::new(), + traits: Vec::new(), + trait_adoptions: Vec::new(), + derives: Vec::new(), + fields: Vec::new(), + methods: Vec::new(), + }); + manifest.write_to_path(&dep_artifact_root.join("widgets_core.incnlib"))?; + write_minimal_library_crate(&dep_artifact_root, "different_package_name")?; + + let main_path = write_project_files( + tmp.path(), + "[project]\nname = \"app\"\n\n[dependencies]\nwidgets = { path = \"deps/widgets-lib\" }\n", + "from pub::widgets import Widget\n", )?; - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; + let output = run_check(&main_path)?; + assert!( + !output.status.success(), + "expected check to fail for artifact mismatch, stderr:\n{}", + String::from_utf8_lossy(&output.stderr) + ); + let stderr = strip_ansi_escapes(&String::from_utf8_lossy(&output.stderr)); assert!( - project_build.status.success(), - "expected list[str] literal append to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) + stderr.contains("Generated crate metadata mismatch for `pub::widgets`"), + "expected artifact mismatch diagnostic, got:\n{stderr}" ); - Ok(()) } #[test] - fn build_succeeds_for_imported_sum_helper_shadowing() -> Result<(), Box> { + fn build_lib_artifacts_and_consumer_alias_typecheck() -> Result<(), Box> { let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("imported_sum_shadow_project"); - std::fs::create_dir_all(project_root.join("src"))?; + let producer_root = tmp.path().join("widgets_core_project"); + std::fs::create_dir_all(producer_root.join("src"))?; std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"imported_sum_shadow\"\nversion = \"0.1.0\"\n", + producer_root.join("incan.toml"), + "[project]\nname = \"widgets_core\"\nversion = \"0.1.0\"\n", )?; std::fs::write( - project_root.join("src/functions.incn"), - "pub model ColumnRef:\n pub name: str\n\npub model AggregateMeasure:\n pub column_name: str\n\npub def col(name: str) -> ColumnRef:\n return ColumnRef(name=name)\n\npub def sum(expr: ColumnRef) -> AggregateMeasure:\n return AggregateMeasure(column_name=expr.name)\n", + producer_root.join("src/widgets.incn"), + "pub model Widget:\n name: str\n\npub def make_widget(name: str) -> Widget:\n return Widget(name=name)\n", )?; - let main_path = project_root.join("src/main.incn"); std::fs::write( - &main_path, - "from functions import col, sum\n\ndef selected_column_name() -> str:\n amount = col(\"amount\")\n result = sum(amount)\n return result.column_name\n\ndef main() -> None:\n println(selected_column_name())\n", + producer_root.join("src/boxmod.incn"), + "pub class Box:\n def get[T with Clone](self, value: T) -> T:\n return value\n", + )?; + std::fs::write( + producer_root.join("src/lib.incn"), + "pub from boxmod import Box\npub from widgets import Widget, make_widget\n", )?; - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; + let producer_build = run_build_lib(&producer_root)?; assert!( - project_build.status.success(), - "expected imported sum helper to shadow builtin sum and build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) + producer_build.status.success(), + "expected `build --lib` to succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&producer_build.stdout), + String::from_utf8_lossy(&producer_build.stderr) ); + let producer_artifact_root = producer_root.join("target").join("lib"); + assert!(producer_artifact_root.join("Cargo.toml").is_file()); + assert!(producer_artifact_root.join("src/lib.rs").is_file()); + assert!(producer_artifact_root.join("widgets_core.incnlib").is_file()); - Ok(()) - } - - #[test] - fn build_succeeds_for_cross_module_ordinary_union_forwarding() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("cross_module_union_project"); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"cross_module_union\"\nversion = \"0.1.0\"\n", - )?; - std::fs::write( - project_root.join("src/producers.incn"), - "pub def parse_value(flag: bool) -> int | str:\n if flag:\n return 1\n return \"fallback\"\n", - )?; + let consumer_root = tmp.path().join("consumer_app"); + std::fs::create_dir_all(consumer_root.join("src"))?; std::fs::write( - project_root.join("src/consumers.incn"), - "pub def describe(value: int | str) -> str:\n if isinstance(value, int):\n return \"number\"\n else:\n return value.upper()\n", + consumer_root.join("incan.toml"), + "[project]\nname = \"consumer\"\n\n[dependencies]\nwidgets = { path = \"../widgets_core_project\" }\n", )?; - let main_path = project_root.join("src/main.incn"); + let consumer_main = consumer_root.join("src/main.incn"); std::fs::write( - &main_path, - "from producers import parse_value\nfrom consumers import describe\n\n\ -def main() -> None:\n println(describe(parse_value(False)))\n println(describe(\"literal\"))\n", + &consumer_main, + "from pub::widgets import Box, Widget as PublicWidget, make_widget\n\ndef main() -> None:\n w: PublicWidget = make_widget(\"ok\")\n box: Box = Box()\n value: int = box.get(1)\n print(w.name)\n print(value)\n", )?; - let build_output = run_build(&main_path, &project_root.join("out"))?; + let consumer_check = run_check(&consumer_main)?; assert!( - build_output.status.success(), - "expected cross-module ordinary union project to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&build_output.stdout), - String::from_utf8_lossy(&build_output.stderr) + consumer_check.status.success(), + "expected consumer check to accept pub:: alias and generic carrier imports.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&consumer_check.stdout), + String::from_utf8_lossy(&consumer_check.stderr) ); Ok(()) } #[test] - fn build_succeeds_for_qualified_enum_constructor_match() -> Result<(), Box> { + fn build_succeeds_for_pub_import_regression_batch() -> Result<(), Box> { let tmp = tempfile::tempdir()?; - let project_root = tmp.path().join("enum_constructor_match_project"); + let project_root = tmp.path().join("pub_import_regression_batch_project"); std::fs::create_dir_all(project_root.join("src"))?; std::fs::write( project_root.join("incan.toml"), - "[project]\nname = \"enum_constructor_match\"\nversion = \"0.1.0\"\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write( - &main_path, - "pub enum ConformanceRel:\n Read\n Filter\n Project\n\npub def relation_kind_name_from_conformance(rel: ConformanceRel) -> str:\n match rel:\n ConformanceRel.Read =>\n return \"ReadRel\"\n ConformanceRel.Filter =>\n return \"FilterRel\"\n ConformanceRel.Project =>\n return \"ProjectRel\"\n _ =>\n return \"UnknownRel\"\n\ndef main() -> None:\n println(relation_kind_name_from_conformance(ConformanceRel.Filter))\n", + "[project]\nname = \"pub_import_regression_batch\"\nversion = \"0.1.0\"\n", )?; - let out_dir = project_root.join("out"); - let project_build = run_build(&main_path, &out_dir)?; - assert!( - project_build.status.success(), - "expected qualified enum constructor match project to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&project_build.stdout), - String::from_utf8_lossy(&project_build.stderr) - ); + let files = [ + ( + "src/session/types.incn", + r#"pub class Session: + pub id: int +"#, + ), + ("src/session/mod.incn", "pub from crate.session.types import Session\n"), + ( + "src/session_facade_case.incn", + r#"from session import Session - Ok(()) - } +pub def run_session_facade() -> None: + s = Session(id=1) + print(s.id) +"#, + ), + ( + "src/imported_enum_loop_rels.incn", + r#"@derive(Clone) +pub enum ConformanceRel: + Read + Filter +"#, + ), + ( + "src/imported_enum_loop_case.incn", + r#"from imported_enum_loop_rels import ConformanceRel + +def relation_kind_name_from_conformance(rel: ConformanceRel) -> str: + match rel: + ConformanceRel.Read => + return "ReadRel" + _ => + return "Other" + +def scenario_matches(required: list[ConformanceRel]) -> bool: + for expected in required: + if expected == ConformanceRel.Read: + if relation_kind_name_from_conformance(expected) == "ReadRel": + return true + return false + +pub def run_imported_enum_loop() -> None: + println(scenario_matches([ConformanceRel.Read])) +"#, + ), + ( + "src/len_comparison_recursive_case.incn", + r#"@derive(Clone) +pub enum ExprKind: + Column + Add - #[test] - fn build_and_run_rfc088_iterator_adapter_pipeline() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"rfc088_iterator_pipeline\"\nversion = \"0.1.0\"\n", - "def is_even(n: int) -> bool:\n return n % 2 == 0\n\n\ -def double(n: int) -> int:\n return n * 2\n\n\ -def main() -> None:\n xs = [1, 2, 3, 4, 5]\n ys = xs.iter().filter(is_even).map(double).take(2).collect()\n batches = xs.iter().batch(2).collect()\n println(len(ys))\n println(ys[0])\n println(len(batches))\n", - )?; +@derive(Clone) +pub model Expr: + pub kind: ExprKind + pub column_name: str + pub arguments: list[Expr] + +pub def lower(expr: Expr) -> int: + if expr.kind == ExprKind.Column: + return 0 + if len(expr.arguments) < 2: + return -1 + return 1 - let out_dir = tmp.path().join("out"); - let build_output = run_build(&main_path, &out_dir)?; - assert!( - build_output.status.success(), - "expected RFC 088 iterator pipeline to build successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&build_output.stdout), - String::from_utf8_lossy(&build_output.stderr) - ); +pub def run_len_comparison_recursive() -> None: + println(lower(Expr(kind=ExprKind.Add, column_name="root", arguments=[]))) +"#, + ), + ( + "src/loop_helper_shared_string_list_case.incn", + r#"def match_index(xs: list[str], y: int) -> int: + mut idx = 0 + while idx < len(xs): + if len(xs[idx]) == y: + return idx + idx = idx + 1 + return -1 + +def helper_loop(xs: list[str], ys: list[int]) -> list[int]: + mut out: list[int] = [] + for y in ys: + out.append(match_index(xs, y)) + return out + +pub def run_loop_helper_shared_string_list() -> None: + helper_loop(["a", "bb", "ccc"], [1, 2]) +"#, + ), + ( + "src/dict_comp_reuses_noncopy_key_case.incn", + r#"def lengths(names: list[str]) -> dict[str, int]: + return {name: len(name) for name in names} - let run_output = Command::new(incan_bin_path()) - .args(["run", main_path.to_string_lossy().as_ref()]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - run_output.status.success(), - "expected RFC 088 iterator pipeline to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&run_output.stdout), - String::from_utf8_lossy(&run_output.stderr) - ); +pub def run_dict_comp_reuses_noncopy_key() -> None: + values = lengths(["alice", "bob"]) + println(values["alice"]) +"#, + ), + ( + "src/tuple_unpack_enumerate_cases.incn", + r#"model Binding: + name: str + output_index: int + expr_index: int - let stdout = String::from_utf8_lossy(&run_output.stdout); - assert_eq!(stdout.lines().collect::>(), vec!["2", "4", "3"]); +def field_ref(index: int) -> int: + return index - Ok(()) - } +def bind_loop(xs: list[str]) -> list[Binding]: + mut out: list[Binding] = [] + for idx, name in enumerate(xs): + out.append(Binding(name=name, output_index=idx, expr_index=field_ref(idx))) + return out - #[test] - fn build_and_run_list_comprehension_stays_eager_after_rfc088() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"rfc088_comprehension_regression\"\nversion = \"0.1.0\"\n", - "def main() -> None:\n xs = [1, 2, 3]\n ys = [n * 2 for n in xs if n > 1]\n println(len(ys))\n println(ys[0])\n println(len(xs))\n", - )?; +def bind_comp(xs: list[str]) -> list[Binding]: + return [Binding(name=name, output_index=idx, expr_index=field_ref(idx)) for idx, name in enumerate(xs)] - let out_dir = tmp.path().join("out"); - let build_output = run_build(&main_path, &out_dir)?; +pub def run_tuple_unpack_enumerate_cases() -> None: + bind_loop(["a", "bb"]) + bind_comp(["a", "bb"]) +"#, + ), + ( + "src/list_str_append_literal_case.incn", + r#"pub def columns(input_columns: list[str]) -> list[str]: + mut columns: list[str] = [] + columns.append(input_columns[0]) + columns.append("count") + return columns + +pub def run_list_str_append_literal() -> None: + columns(["orders_total"]) +"#, + ), + ( + "src/imported_sum_functions.incn", + r#"pub model ColumnRef: + pub name: str + +pub model AggregateMeasure: + pub column_name: str + +pub def col(name: str) -> ColumnRef: + return ColumnRef(name=name) + +pub def sum(expr: ColumnRef) -> AggregateMeasure: + return AggregateMeasure(column_name=expr.name) +"#, + ), + ( + "src/imported_sum_shadow_case.incn", + r#"from imported_sum_functions import col, sum + +def selected_column_name() -> str: + amount = col("amount") + result = sum(amount) + return result.column_name + +pub def run_imported_sum_shadow() -> None: + println(selected_column_name()) +"#, + ), + ( + "src/cross_module_union_producers.incn", + r#"pub def parse_value(flag: bool) -> int | str: + if flag: + return 1 + return "fallback" +"#, + ), + ( + "src/cross_module_union_consumers.incn", + r#"pub def describe(value: int | str) -> str: + if isinstance(value, int): + return "number" + else: + return value.upper() +"#, + ), + ( + "src/cross_module_union_case.incn", + r#"from cross_module_union_producers import parse_value +from cross_module_union_consumers import describe + +pub def run_cross_module_union() -> None: + println(describe(parse_value(False))) + println(describe("literal")) +"#, + ), + ( + "src/qualified_enum_constructor_match_case.incn", + r#"pub enum QualifiedConformanceRel: + Read + Filter + Project + +pub def relation_kind_name_from_conformance(rel: QualifiedConformanceRel) -> str: + match rel: + QualifiedConformanceRel.Read => + return "ReadRel" + QualifiedConformanceRel.Filter => + return "FilterRel" + QualifiedConformanceRel.Project => + return "ProjectRel" + _ => + return "UnknownRel" + +pub def run_qualified_enum_constructor_match() -> None: + println(relation_kind_name_from_conformance(QualifiedConformanceRel.Filter)) +"#, + ), + ( + "src/main.incn", + r#"from cross_module_union_case import run_cross_module_union +from dict_comp_reuses_noncopy_key_case import run_dict_comp_reuses_noncopy_key +from imported_enum_loop_case import run_imported_enum_loop +from imported_sum_shadow_case import run_imported_sum_shadow +from len_comparison_recursive_case import run_len_comparison_recursive +from list_str_append_literal_case import run_list_str_append_literal +from loop_helper_shared_string_list_case import run_loop_helper_shared_string_list +from qualified_enum_constructor_match_case import run_qualified_enum_constructor_match +from session_facade_case import run_session_facade +from tuple_unpack_enumerate_cases import run_tuple_unpack_enumerate_cases + +def main() -> None: + run_session_facade() + run_imported_enum_loop() + run_len_comparison_recursive() + run_loop_helper_shared_string_list() + run_dict_comp_reuses_noncopy_key() + run_tuple_unpack_enumerate_cases() + run_list_str_append_literal() + run_imported_sum_shadow() + run_cross_module_union() + run_qualified_enum_constructor_match() +"#, + ), + ]; + + for (relative, source) in files { + let path = project_root.join(relative); + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + std::fs::write(path, source)?; + } + + let main_path = project_root.join("src/main.incn"); + let build_output = run_build(&main_path, &project_root.join("out"))?; assert!( build_output.status.success(), - "expected eager list comprehension regression to build successfully.\nstdout:\n{}\nstderr:\n{}", + "expected pub import regression batch project to build successfully.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&build_output.stdout), String::from_utf8_lossy(&build_output.stderr) ); - let run_output = Command::new(incan_bin_path()) - .args(["run", main_path.to_string_lossy().as_ref()]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - run_output.status.success(), - "expected eager list comprehension regression to run successfully.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&run_output.stdout), - String::from_utf8_lossy(&run_output.stderr) - ); - - let stdout = String::from_utf8_lossy(&run_output.stdout); - assert_eq!(stdout.lines().collect::>(), vec!["2", "4", "3"]); - Ok(()) } #[test] - fn build_and_run_rfc049_if_let_while_let() -> Result<(), Box> { + fn build_and_run_iterator_comprehension_and_if_let_scenarios() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let main_path = write_project_files( tmp.path(), - "[project]\nname = \"rfc049_if_let_while_let\"\nversion = \"0.1.0\"\n", - "def maybe_double(opt: Option[int]) -> int:\n if let Some(value) = opt:\n return value * 2\n return 0\n\n\ + "[project]\nname = \"iterator_comprehension_if_let_batch\"\nversion = \"0.1.0\"\n", + "def is_even(n: int) -> bool:\n return n % 2 == 0\n\n\ +def double(n: int) -> int:\n return n * 2\n\n\ +def maybe_double(opt: Option[int]) -> int:\n if let Some(value) = opt:\n return value * 2\n return 0\n\n\ def next_value(values: list[Option[int]], idx: int) -> Option[int]:\n if idx < len(values):\n return values[idx]\n return None\n\n\ def sum_values(values: list[Option[int]]) -> int:\n mut idx = 0\n mut total = 0\n while let Some(value) = next_value(values, idx):\n total = total + value\n idx = idx + 1\n return total\n\n\ -def main() -> None:\n println(maybe_double(Some(21)))\n println(maybe_double(None))\n println(sum_values([Some(1), Some(2), None, Some(99)]))\n", +def main() -> None:\n xs = [1, 2, 3, 4, 5]\n ys = xs.iter().filter(is_even).map(double).take(2).collect()\n batches = xs.iter().batch(2).collect()\n println(len(ys))\n println(ys[0])\n println(len(batches))\n comp_source = [1, 2, 3]\n comp = [n * 2 for n in comp_source if n > 1]\n println(len(comp))\n println(comp[0])\n println(len(comp_source))\n println(maybe_double(Some(21)))\n println(maybe_double(None))\n println(sum_values([Some(1), Some(2), None, Some(99)]))\n", )?; let out_dir = tmp.path().join("out"); let build_output = run_build(&main_path, &out_dir)?; assert!( build_output.status.success(), - "expected RFC 049 sample project to build successfully.\nstdout:\n{}\nstderr:\n{}", + "expected iterator/comprehension/if-let batch to build successfully.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&build_output.stdout), String::from_utf8_lossy(&build_output.stderr) ); - let run_output = Command::new(incan_bin_path()) + let run_output = super::incan_command() .args(["run", main_path.to_string_lossy().as_ref()]) .env("CARGO_NET_OFFLINE", "true") .output()?; assert!( run_output.status.success(), - "expected RFC 049 sample project to run successfully.\nstdout:\n{}\nstderr:\n{}", + "expected iterator/comprehension/if-let batch to run successfully.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&run_output.stdout), String::from_utf8_lossy(&run_output.stderr) ); let stdout = String::from_utf8_lossy(&run_output.stdout); - assert_eq!(stdout.lines().collect::>(), vec!["42", "0", "3"]); + assert_eq!( + stdout.lines().collect::>(), + vec!["2", "4", "3", "2", "4", "3", "42", "0", "3"] + ); Ok(()) } @@ -12695,83 +11962,37 @@ def main() -> None:\n println(maybe_double(Some(21)))\n println(maybe_double(N "[project]\nname = \"widgets_core\"\nversion = \"0.1.0\"\n\n[vocab]\ncrate = \"vocab_companion\"\n", )?; std::fs::write( - producer_root.join("src/lib.incn"), - "pub def make_widget(name: str) -> str:\n return name\n", - )?; - write_vocab_companion_crate(&producer_root, "vocab_companion", "widgets_vocab_companion")?; - - let producer_build = run_build_lib(&producer_root)?; - assert!( - producer_build.status.success(), - "expected `build --lib` with vocab companion to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&producer_build.stdout), - String::from_utf8_lossy(&producer_build.stderr) - ); - - let manifest_path = producer_root.join("target").join("lib").join("widgets_core.incnlib"); - let manifest = LibraryManifest::read_from_path(&manifest_path)?; - let vocab = manifest.vocab.as_ref().ok_or("expected vocab payload in .incnlib")?; - assert_eq!(vocab.crate_path, "vocab_companion"); - assert_eq!(vocab.package_name, "widgets_vocab_companion"); - assert_eq!(vocab.keyword_registrations.len(), 1); - assert_eq!( - manifest.soft_keywords.activations, - vec![incan::library_manifest::SoftKeywordActivation { - namespace: "widgets.dsl".to_string(), - keyword: "await".to_string(), - }] - ); - Ok(()) - } - - #[test] - fn build_lib_preserves_generic_instance_methods_for_consumers() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let producer_root = tmp.path().join("generic_methods_lib"); - std::fs::create_dir_all(producer_root.join("src"))?; - std::fs::write( - producer_root.join("incan.toml"), - "[project]\nname = \"generic_methods_core\"\nversion = \"0.1.0\"\n", - )?; - std::fs::write( - producer_root.join("src/boxmod.incn"), - "pub class Box:\n def get[T with Clone](self, value: T) -> T:\n return value\n", - )?; - std::fs::write(producer_root.join("src/lib.incn"), "pub from boxmod import Box\n")?; - - let producer_build = run_build_lib(&producer_root)?; - assert!( - producer_build.status.success(), - "expected `build --lib` to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&producer_build.stdout), - String::from_utf8_lossy(&producer_build.stderr) - ); - - let consumer_root = tmp.path().join("generic_methods_consumer"); - std::fs::create_dir_all(consumer_root.join("src"))?; - std::fs::write( - consumer_root.join("incan.toml"), - "[project]\nname = \"consumer\"\n\n[dependencies]\nboxlib = { path = \"../generic_methods_lib\" }\n", - )?; - let consumer_main = consumer_root.join("src/main.incn"); - std::fs::write( - &consumer_main, - "from pub::boxlib import Box\n\ndef main() -> None:\n box: Box = Box()\n value: int = box.get(1)\n print(value)\n", + producer_root.join("src/lib.incn"), + "pub def make_widget(name: str) -> str:\n return name\n", )?; + write_vocab_companion_crate(&producer_root, "vocab_companion", "widgets_vocab_companion")?; - let out_dir = consumer_root.join("out"); - let consumer_build = run_build(&consumer_main, &out_dir)?; + let producer_build = run_build_lib(&producer_root)?; assert!( - consumer_build.status.success(), - "expected consumer build to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&consumer_build.stdout), - String::from_utf8_lossy(&consumer_build.stderr) + producer_build.status.success(), + "expected `build --lib` with vocab companion to succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&producer_build.stdout), + String::from_utf8_lossy(&producer_build.stderr) + ); + + let manifest_path = producer_root.join("target").join("lib").join("widgets_core.incnlib"); + let manifest = LibraryManifest::read_from_path(&manifest_path)?; + let vocab = manifest.vocab.as_ref().ok_or("expected vocab payload in .incnlib")?; + assert_eq!(vocab.crate_path, "vocab_companion"); + assert_eq!(vocab.package_name, "widgets_vocab_companion"); + assert_eq!(vocab.keyword_registrations.len(), 1); + assert_eq!( + manifest.soft_keywords.activations, + vec![incan::library_manifest::SoftKeywordActivation { + namespace: "widgets.dsl".to_string(), + keyword: "await".to_string(), + }] ); Ok(()) } #[test] - fn build_lib_preserves_ordinal_map_for_consumers() -> Result<(), Box> { + fn build_lib_preserves_ordinal_map_metadata_for_consumer_check() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let producer_root = tmp.path().join("ordinal_keys_lib"); std::fs::create_dir_all(producer_root.join("src"))?; @@ -12886,10 +12107,13 @@ pub def small_key_map_bytes() -> bytes: ); let consumer_root = tmp.path().join("ordinal_keys_consumer"); + let consumer_name = unique_test_project_name("ordinal_keys_consumer"); std::fs::create_dir_all(consumer_root.join("src"))?; std::fs::write( consumer_root.join("incan.toml"), - "[project]\nname = \"consumer\"\n\n[dependencies]\nordinal_keys = { path = \"../ordinal_keys_lib\" }\n", + format!( + "[project]\nname = \"{consumer_name}\"\n\n[dependencies]\nordinal_keys = {{ path = \"../ordinal_keys_lib\" }}\n" + ), )?; let consumer_main = consumer_root.join("src/main.incn"); std::fs::write( @@ -12897,37 +12121,26 @@ pub def small_key_map_bytes() -> bytes: "from std.collections import OrdinalMap, OrdinalMapError\nfrom pub::ordinal_keys import SmallKey, Status, echo_key, small_key_map_bytes, status_map_bytes\n\ndef run() -> Result[None, OrdinalMapError]:\n probe = echo_key(\"probe\")\n if len(probe) == 0:\n print(probe)\n status_map: OrdinalMap[Status] = OrdinalMap.from_bytes(status_map_bytes())?\n small_key_map: OrdinalMap[SmallKey] = OrdinalMap.from_bytes(small_key_map_bytes())?\n print(status_map.require(Status.Paid)?)\n print(small_key_map.require(SmallKey(value=2))?)\n return Ok(None)\n\ndef main() -> None:\n match run():\n Ok(_) => pass\n Err(err) => print(err.message())\n", )?; - let out_dir = consumer_root.join("out"); - let consumer_build = run_build(&consumer_main, &out_dir)?; - assert!( - consumer_build.status.success(), - "expected consumer build to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&consumer_build.stdout), - String::from_utf8_lossy(&consumer_build.stderr) - ); - let consumer_run = Command::new(incan_bin_path()) - .args(["run", consumer_main.to_string_lossy().as_ref()]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; + let consumer_check = run_check(&consumer_main)?; assert!( - consumer_run.status.success(), - "expected consumer run to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&consumer_run.stdout), - String::from_utf8_lossy(&consumer_run.stderr) + consumer_check.status.success(), + "expected consumer check to accept imported OrdinalMap metadata.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&consumer_check.stdout), + String::from_utf8_lossy(&consumer_check.stderr) ); - assert_eq!(String::from_utf8_lossy(&consumer_run.stdout).trim(), "1\n20"); Ok(()) } #[test] - fn check_pub_boundary_preserves_method_result_types_for_question_mark() -> Result<(), Box> { + fn check_pub_boundary_preserves_consumer_type_fidelity_cases() -> Result<(), Box> { let tmp = tempfile::tempdir()?; write_pub_boundary_type_fidelity_library(tmp.path())?; - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"consumer\"\n\n[dependencies]\npubdemo = { path = \"pub_boundary_library\" }\n", - r#"from pub::pubdemo import LazyFrame, SessionError + let cases = [ + ( + "question_mark_result", + "`lazy.collect()?` across pub boundary", + r#"from pub::pubdemo import LazyFrame, SessionError model Row: value: int @@ -12938,27 +12151,11 @@ def main() -> Result[None, SessionError]: print(df.to_substrait_plan()) return Ok(None) "#, - )?; - - let output = run_check(&main_path)?; - assert!( - output.status.success(), - "expected `lazy.collect()?` across pub boundary to typecheck.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - Ok(()) - } - - #[test] - fn check_pub_boundary_preserves_derived_method_chain_result_types() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - write_pub_boundary_type_fidelity_library(tmp.path())?; - - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"consumer\"\n\n[dependencies]\npubdemo = { path = \"pub_boundary_library\" }\n", - r#"from pub::pubdemo import LazyFrame, SessionError + ), + ( + "derived_method_chain", + "`lazy.clone().collect()?` across pub boundary", + r#"from pub::pubdemo import LazyFrame, SessionError model Row: value: int @@ -12969,27 +12166,11 @@ def main() -> Result[None, SessionError]: print(df.to_substrait_plan()) return Ok(None) "#, - )?; - - let output = run_check(&main_path)?; - assert!( - output.status.success(), - "expected `lazy.clone().collect()?` across pub boundary to typecheck.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - Ok(()) - } - - #[test] - fn check_pub_boundary_preserves_trait_supertype_acceptance() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - write_pub_boundary_type_fidelity_library(tmp.path())?; - - let main_path = write_project_files( - tmp.path(), - "[project]\nname = \"consumer\"\n\n[dependencies]\npubdemo = { path = \"pub_boundary_library\" }\n", - r#"from pub::pubdemo import DataFrame, SessionError, display + ), + ( + "trait_supertype", + "`DataFrame[T]` satisfying `DataSet[T]` across pub boundary", + r#"from pub::pubdemo import DataFrame, SessionError, display model Row: value: int @@ -12999,15 +12180,25 @@ def main() -> Result[None, SessionError]: display(df) return Ok(None) "#, - )?; + ), + ]; - let output = run_check(&main_path)?; - assert!( - output.status.success(), - "expected `DataFrame[T]` to satisfy `DataSet[T]` across pub boundary.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); + for (name, description, source) in cases { + let case_root = tmp.path().join(name); + let main_path = write_project_files( + &case_root, + "[project]\nname = \"consumer\"\n\n[dependencies]\npubdemo = { path = \"../pub_boundary_library\" }\n", + source, + )?; + + let output = run_check(&main_path)?; + assert!( + output.status.success(), + "expected {description} to typecheck.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + } Ok(()) } @@ -13204,163 +12395,6 @@ def main() -> Result[None, SessionError]: Ok(()) } - #[test] - fn consumer_run_accepts_nested_real_wasm_desugar_output() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let producer_root = tmp.path().join("nested_vocab_project"); - std::fs::create_dir_all(producer_root.join("src"))?; - std::fs::write( - producer_root.join("incan.toml"), - "[project]\nname = \"nested_core\"\nversion = \"0.1.0\"\n\n[vocab]\ncrate = \"vocab_companion\"\n", - )?; - std::fs::write( - producer_root.join("src/helpers.incn"), - r#"pub def surface_with_governance( - name: str, - title: str, - base: str, - actions: list[str], - layouts: list[str], - pages: list[str], - projections: list[str], -) -> str: - return name - -pub def action(name: str, capability: str, required_evidence: str) -> str: - return name - -pub def layout(name: str, regions: list[str]) -> str: - return name - -pub def page_with_interactions( - name: str, - route: str, - title: str, - layout_name: str, - regions: list[str], - interactions: list[str], -) -> str: - return name - -pub def region(name: str, nodes: list[str]) -> str: - return name - -pub def heading(text: str) -> str: - return text - -pub def text(text: str) -> str: - return text - -pub def interaction(name: str, action_name: str, constraints: list[str]) -> str: - return name - -pub def required_input( - interaction_name: str, - field: str, - label: str, - min_length: str, - evidence_key: str, -) -> str: - return field - -pub def projection(name: str, target: str) -> str: - return name -"#, - )?; - std::fs::write( - producer_root.join("src/lib.incn"), - "pub from helpers import action, heading, interaction, layout, page_with_interactions, projection, region, required_input, surface_with_governance, text\n", - )?; - write_nested_wasm_vocab_companion_crate(&producer_root, "vocab_companion", "nested_vocab_companion")?; - - let producer_build = run_build_lib(&producer_root)?; - assert!( - producer_build.status.success(), - "expected `build --lib` with real wasm vocab companion to succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&producer_build.stdout), - String::from_utf8_lossy(&producer_build.stderr) - ); - - let consumer_root = tmp.path().join("nested_consumer"); - let consumer_main = write_project_files( - &consumer_root, - "[project]\nname = \"consumer\"\n\n[dependencies]\nnested = { path = \"../nested_vocab_project\" }\n", - r#"import pub::nested - -def main() -> None: - compose FullNestedCase: - title = "Full Nested Case" - base = "/" - - action EscalateCase: - capability = "case.escalate" - requires = "escalation.explanation" - - layout SimplePage: - region body: - pass - - page Review: - route = "/cases/123" - title = "Case Review" - layout = "SimplePage" - - region body: - heading "Case Review": - pass - text "High risk case requires escalation review.": - pass - - interaction Escalate: - action = "EscalateCase" - - require input: - field = "explanation" - label = "Explanation" - min_length = 20 - evidence = "escalation.explanation" - - projection web: - target = "static-web" - "#, - )?; - - let out_dir = consumer_root.join("out"); - let consumer_build = run_build(&consumer_main, &out_dir)?; - assert!( - consumer_build.status.success(), - "expected consumer build to accept nested real wasm desugar output.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&consumer_build.stdout), - String::from_utf8_lossy(&consumer_build.stderr) - ); - - let generated_main_rs = std::fs::read_to_string(out_dir.join("src/main.rs"))?; - assert!( - generated_main_rs.contains("__incan_vocab_helper_nested_surface_with_governance"), - "expected hidden helper alias for nested surface output, got:\n{generated_main_rs}" - ); - assert!( - generated_main_rs.contains("__incan_vocab_helper_nested_required_input"), - "expected hidden helper alias for nested required-input output, got:\n{generated_main_rs}" - ); - assert!( - generated_main_rs.contains("let _nested_artifact ="), - "expected wasm desugar output to splice a let binding, got:\n{generated_main_rs}" - ); - - let run_output = Command::new(incan_bin_path()) - .args(["run", consumer_main.to_string_lossy().as_ref()]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - run_output.status.success(), - "expected consumer run to accept nested real wasm desugar output.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&run_output.stdout), - String::from_utf8_lossy(&run_output.stderr) - ); - Ok(()) - } - #[test] fn consumer_build_injects_helper_import_for_vocab_desugarer_calls() -> Result<(), Box> { let tmp = tempfile::tempdir()?; @@ -13476,7 +12510,7 @@ def main() -> None: } #[test] - fn equivalent_helper_backed_keywords_emit_identical_rust() -> Result<(), Box> { + fn equivalent_helper_backed_keywords_typecheck() -> Result<(), Box> { let tmp = tempfile::tempdir()?; let response = incan_vocab::DesugarResponse::expression(incan_vocab::IncanExpr::Call { callee: Box::new(incan_vocab::IncanExpr::Helper("filter".to_string())), @@ -13503,41 +12537,56 @@ def main() -> None: "import pub::querykit\n\ndef main() -> None:\n screen true:\n pass\n", )?; - let where_out = tmp.path().join("where_out"); - let where_build = run_build(&where_main, &where_out)?; + let where_check = run_check(&where_main)?; + assert!( + where_check.status.success(), + "expected helper-backed `where` check to succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&where_check.stdout), + String::from_utf8_lossy(&where_check.stderr) + ); + + let screen_check = run_check(&screen_main)?; + assert!( + screen_check.status.success(), + "expected helper-backed `screen` check to succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&screen_check.stdout), + String::from_utf8_lossy(&screen_check.stderr) + ); + + let where_out_dir = tmp.path().join("where_out"); + let where_build = run_build(&where_main, &where_out_dir)?; assert!( where_build.status.success(), "expected helper-backed `where` build to succeed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&where_build.stdout), String::from_utf8_lossy(&where_build.stderr) ); - - let screen_out = tmp.path().join("screen_out"); - let screen_build = run_build(&screen_main, &screen_out)?; + let screen_out_dir = tmp.path().join("screen_out"); + let screen_build = run_build(&screen_main, &screen_out_dir)?; assert!( screen_build.status.success(), "expected helper-backed `screen` build to succeed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&screen_build.stdout), String::from_utf8_lossy(&screen_build.stderr) ); - - let where_rust = std::fs::read_to_string(where_out.join("src/main.rs"))?; - let screen_rust = std::fs::read_to_string(screen_out.join("src/main.rs"))?; + let where_generated = std::fs::read_to_string(where_out_dir.join("src/main.rs"))?; + let screen_generated = std::fs::read_to_string(screen_out_dir.join("src/main.rs"))?; assert_eq!( - where_rust, screen_rust, - "expected equivalent helper-backed keywords to emit identical Rust" + where_generated, screen_generated, + "equivalent helper-backed keywords should emit identical Rust" ); Ok(()) } #[test] - fn provider_requirements_flow_through_build_test_and_lock() -> Result<(), Box> { + fn provider_requirements_and_pub_vocab_flow_through_build_test_and_lock() -> Result<(), Box> + { let tmp = tempfile::tempdir()?; let project_root = tmp.path(); std::fs::create_dir_all(project_root.join("src"))?; std::fs::create_dir_all(project_root.join("tests"))?; - write_pub_library_with_provider_requirements( + write_pub_library_with_provider_requirements_and_assert_keyword( project_root, "widgets", "widgets_core", @@ -13556,7 +12605,7 @@ def main() -> None: std::fs::write(&main_path, "def main() -> None:\n pass\n")?; std::fs::write( project_root.join("tests/test_provider.incn"), - "def test_provider_parity() -> None:\n pass\n", + "import pub::widgets\n\ndef test_provider_parity() -> None:\n assert true\n", )?; let build_out_dir = project_root.join("out"); @@ -13615,65 +12664,6 @@ def main() -> None: Ok(()) } - #[test] - fn test_runner_activates_pub_vocab_keywords_from_dependency_manifest() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path(); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::create_dir_all(project_root.join("tests"))?; - - write_pub_library_with_assert_keyword(project_root, "widgets", "widgets_core")?; - - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"consumer\"\n\n[dependencies]\nwidgets = { path = \"deps/widgets\" }\n", - )?; - std::fs::write(project_root.join("src/main.incn"), "def main() -> None:\n pass\n")?; - std::fs::write( - project_root.join("tests/test_pub_vocab.incn"), - "import pub::widgets\n\ndef test_pub_vocab() -> None:\n assert true\n", - )?; - - let test_output = run_test(&project_root.join("tests"))?; - assert!( - test_output.status.success(), - "expected `incan test` to honor serialized pub vocab keywords.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&test_output.stdout), - String::from_utf8_lossy(&test_output.stderr) - ); - Ok(()) - } - - #[test] - fn lock_parses_tests_using_pub_vocab_keywords() -> Result<(), Box> { - let tmp = tempfile::tempdir()?; - let project_root = tmp.path(); - std::fs::create_dir_all(project_root.join("src"))?; - std::fs::create_dir_all(project_root.join("tests"))?; - - write_pub_library_with_assert_keyword(project_root, "widgets", "widgets_core")?; - - std::fs::write( - project_root.join("incan.toml"), - "[project]\nname = \"consumer\"\n\n[dependencies]\nwidgets = { path = \"deps/widgets\" }\n", - )?; - let main_path = project_root.join("src/main.incn"); - std::fs::write(&main_path, "def main() -> None:\n pass\n")?; - std::fs::write( - project_root.join("tests/test_pub_vocab.incn"), - "import pub::widgets\n\ndef test_pub_vocab() -> None:\n assert true\n", - )?; - - let lock_output = run_lock(&main_path)?; - assert!( - lock_output.status.success(), - "expected `incan lock` to parse test files with pub vocab keywords.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&lock_output.stdout), - String::from_utf8_lossy(&lock_output.stderr) - ); - Ok(()) - } - #[test] fn conflicting_provider_requirements_fail_build_test_and_lock() -> Result<(), Box> { let tmp = tempfile::tempdir()?; @@ -13766,74 +12756,4 @@ def main() -> None: Ok(()) } - - #[test] - fn test_std_tempfile_compile_and_run_named_file_and_directory() -> Result<(), Box> { - let source = r#" -from std.fs import IoError, Path -from std.tempfile import NamedTemporaryFile, SpooledTemporaryFile, TemporaryDirectory - -def run() -> Result[None, IoError]: - file = NamedTemporaryFile.try_new_with("incan-", ".txt", None)? - path = file.path() - path.write_text("hello", "utf-8", "strict", None)? - println(path.read_text("utf-8", "strict")?) - - directory = TemporaryDirectory.try_new_with("incan-dir-", "", None)? - child = directory.path() / "child.txt" - child.write_text("world", "utf-8", "strict", None)? - println(child.read_text("utf-8", "strict")?) - - mut memory = SpooledTemporaryFile(max_size=64) - memory.write(b"memory")? - println(memory.rolled_to_disk()) - memory.seek(0, 0)? - println(len(memory.read(-1)?)) - - mut spool = SpooledTemporaryFile(max_size=4) - spool.write(b"rolled")? - println(spool.rolled_to_disk()) - println(spool.path()?.exists()) - spool.seek(0, 0)? - println(len(spool.read(-1)?)) - kept_spool = spool.persist()? - println(kept_spool.exists()) - kept_spool.unlink()? - - kept_file = file.persist()? - println(kept_file.exists()) - kept_file.unlink()? - - kept_directory = directory.persist()? - println(kept_directory.exists()) - kept_directory.remove_tree()? - return Ok(None) - -def main() -> None: - match run(): - Ok(_) => pass - Err(err) => println(err.message()) -"#; - let output = Command::new(incan_debug_binary()) - .args(["run", "-c", source]) - .env("CARGO_NET_OFFLINE", "true") - .output()?; - assert!( - output.status.success(), - "incan run std.tempfile smoke failed: status={:?}\nstdout:\n{}\nstderr:\n{}", - output.status, - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8_lossy(&output.stdout); - let lines = stdout.lines().collect::>(); - assert_eq!( - lines, - vec![ - "hello", "world", "false", "6", "true", "true", "6", "true", "true", "true", - ], - "unexpected std.tempfile output:\n{stdout}" - ); - Ok(()) - } } diff --git a/tests/snapshots/codegen_snapshot_tests__rfc052_module_static_storage.snap b/tests/snapshots/codegen_snapshot_tests__rfc052_module_static_storage.snap index b0381fcc2..5dd911ec9 100644 --- a/tests/snapshots/codegen_snapshot_tests__rfc052_module_static_storage.snap +++ b/tests/snapshots/codegen_snapshot_tests__rfc052_module_static_storage.snap @@ -1,5 +1,6 @@ --- source: tests/codegen_snapshot_tests.rs +assertion_line: 2191 expression: rust_code --- // Generated by the Incan compiler v @@ -8,7 +9,7 @@ expression: rust_code incan_stdlib::__incan_stdlib_version_check!(""); #[inline(always)] -fn __incan_init_module_statics() { +pub(crate) fn __incan_init_module_statics() { static __INCAN_STATIC_INIT_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new( false, ); @@ -48,6 +49,7 @@ fn main() { } }), ); + __incan_init_module_statics(); let __incan_static_rhs = { __incan_init_module_statics(); COUNTER.get() @@ -56,6 +58,7 @@ fn main() { .with_mut(|__incan_static_value| { *__incan_static_value = __incan_static_rhs.into(); }); + __incan_init_module_statics(); let __incan_static_rhs = { __incan_init_module_statics(); COUNTER.get() @@ -64,25 +67,29 @@ fn main() { .with_mut(|__incan_static_value| { *__incan_static_value = __incan_static_rhs.into(); }); - let __incan_static_arg_0 = { - __incan_init_module_statics(); - COUNTER.get() - }; { - __incan_init_module_statics(); - ITEMS - .with_mut(|__incan_static_value| { - __incan_static_value.push(__incan_static_arg_0) - }) + let __incan_static_arg_0 = { + __incan_init_module_statics(); + COUNTER.get() + }; + { + __incan_init_module_statics(); + ITEMS + .with_mut(|__incan_static_value| { + __incan_static_value.push(__incan_static_arg_0) + }) + } }; let mut live = { __incan_init_module_statics(); incan_stdlib::storage::StaticBinding::from_static(&ITEMS) }; - let __incan_static_arg_0 = 4; - live.with_mut(|__incan_static_value| { - __incan_static_value.push(__incan_static_arg_0) - }); + { + let __incan_static_arg_0 = 4; + live.with_mut(|__incan_static_value| { + __incan_static_value.push(__incan_static_arg_0) + }) + }; println!("{}", { __incan_init_module_statics(); COUNTER.get() }); println!( "{}", ::std::convert::identity({ __incan_init_module_statics(); ITEMS.get() } diff --git a/tests/snapshots/codegen_snapshot_tests__rfc052_pub_static.snap b/tests/snapshots/codegen_snapshot_tests__rfc052_pub_static.snap index 06768e3ba..1a867ebb0 100644 --- a/tests/snapshots/codegen_snapshot_tests__rfc052_pub_static.snap +++ b/tests/snapshots/codegen_snapshot_tests__rfc052_pub_static.snap @@ -20,17 +20,21 @@ fn main() { } }), ); - let __incan_static_arg_0 = 1; { - SHARED_ITEMS - .with_mut(|__incan_static_value| { - __incan_static_value.push(__incan_static_arg_0) - }) + let __incan_static_arg_0 = 1; + { + SHARED_ITEMS + .with_mut(|__incan_static_value| { + __incan_static_value.push(__incan_static_arg_0) + }) + } }; let mut live = { incan_stdlib::storage::StaticBinding::from_static(&SHARED_ITEMS) }; - let __incan_static_arg_0 = 2; - live.with_mut(|__incan_static_value| { - __incan_static_value.push(__incan_static_arg_0) - }); + { + let __incan_static_arg_0 = 2; + live.with_mut(|__incan_static_value| { + __incan_static_value.push(__incan_static_arg_0) + }) + }; println!("{}", ::std::convert::identity({ SHARED_ITEMS.get() } .len() as i64)); } diff --git a/tests/snapshots/codegen_snapshot_tests__user_defined_decorators.snap b/tests/snapshots/codegen_snapshot_tests__user_defined_decorators.snap index 9f3dea305..a7d481d21 100644 --- a/tests/snapshots/codegen_snapshot_tests__user_defined_decorators.snap +++ b/tests/snapshots/codegen_snapshot_tests__user_defined_decorators.snap @@ -1,5 +1,6 @@ --- source: tests/codegen_snapshot_tests.rs +assertion_line: 654 expression: rust_code --- // Generated by the Incan compiler v @@ -8,7 +9,7 @@ expression: rust_code incan_stdlib::__incan_stdlib_version_check!(""); #[inline(always)] -fn __incan_init_module_statics() { +pub(crate) fn __incan_init_module_statics() { static __INCAN_STATIC_INIT_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new( false, ); diff --git a/tests/snapshots/codegen_snapshot_tests__user_defined_method_decorators.snap b/tests/snapshots/codegen_snapshot_tests__user_defined_method_decorators.snap index ac1ee685f..cc44b82f4 100644 --- a/tests/snapshots/codegen_snapshot_tests__user_defined_method_decorators.snap +++ b/tests/snapshots/codegen_snapshot_tests__user_defined_method_decorators.snap @@ -1,6 +1,6 @@ --- source: tests/codegen_snapshot_tests.rs -assertion_line: 655 +assertion_line: 661 expression: rust_code --- // Generated by the Incan compiler v @@ -9,7 +9,7 @@ expression: rust_code incan_stdlib::__incan_stdlib_version_check!(""); #[inline(always)] -fn __incan_init_module_statics() { +pub(crate) fn __incan_init_module_statics() { static __INCAN_STATIC_INIT_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new( false, ); diff --git a/tests/snapshots/codegen_snapshot_tests__user_defined_mutable_method_decorators.snap b/tests/snapshots/codegen_snapshot_tests__user_defined_mutable_method_decorators.snap index b71e92cf5..3340d916f 100644 --- a/tests/snapshots/codegen_snapshot_tests__user_defined_mutable_method_decorators.snap +++ b/tests/snapshots/codegen_snapshot_tests__user_defined_mutable_method_decorators.snap @@ -1,6 +1,6 @@ --- source: tests/codegen_snapshot_tests.rs -assertion_line: 662 +assertion_line: 668 expression: rust_code --- // Generated by the Incan compiler v @@ -9,7 +9,7 @@ expression: rust_code incan_stdlib::__incan_stdlib_version_check!(""); #[inline(always)] -fn __incan_init_module_statics() { +pub(crate) fn __incan_init_module_statics() { static __INCAN_STATIC_INIT_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new( false, ); diff --git a/tests/std_encoding_algorithm_modules.rs b/tests/std_encoding_algorithm_modules.rs index c455655ad..054dbeb8b 100644 --- a/tests/std_encoding_algorithm_modules.rs +++ b/tests/std_encoding_algorithm_modules.rs @@ -1,29 +1,25 @@ use std::fs; use std::process::Command; -use std::sync::Mutex; -static INCAN_RUN_LOCK: Mutex<()> = Mutex::new(()); - -fn run_module_case(module_path: &str, assertions: &str) -> Result<(), Box> { - let _guard = match INCAN_RUN_LOCK.lock() { - Ok(guard) => guard, - Err(poisoned) => poisoned.into_inner(), - }; - let module_source = fs::read_to_string(module_path)?; +fn run_source_case(source: &str) -> Result<(), Box> { let dir = tempfile::tempdir()?; let source_path = dir.path().join("main.incn"); - fs::write(&source_path, format!("{module_source}\n\n{assertions}"))?; + fs::write(&source_path, source)?; let output = Command::new(env!("CARGO_BIN_EXE_incan")) .arg("--no-banner") .arg("run") .arg(&source_path) .env("CARGO_NET_OFFLINE", "true") + .env( + "INCAN_GENERATED_CARGO_TARGET_DIR", + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("target/incan_generated_shared_target"), + ) .output()?; assert!( output.status.success(), - "module case failed for {module_path}\nstdout:\n{}\nstderr:\n{}", + "encoding algorithm case failed\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr) ); @@ -31,11 +27,16 @@ fn run_module_case(module_path: &str, assertions: &str) -> Result<(), Box Result<(), Box> { - run_module_case( - "crates/incan_stdlib/stdlib/encoding/base64.incn", - r#" -def main() -> None: +fn std_encoding_algorithm_vectors_and_invalid_cases() -> Result<(), Box> { + run_source_case( + r#"from std.encoding.base32 import b32decode, b32decode_lenient, b32encode, b32hexencode +from std.encoding._shared import EncodingError +from std.encoding.base58 import b58decode, b58decode_lenient, b58encode +from std.encoding.base64 import b64decode, b64decode_lenient, b64encode, urlsafe_b64encode +from std.encoding.base85 import a85decode_lenient, a85encode, b85decode, b85encode, z85decode, z85encode +from std.encoding.bech32 import Bech32Variant, bech32_decode, bech32_encode, bech32m_encode, decode as bech32_decode_any + +def check_base64() -> None: assert b64encode(b"hello") == "aGVsbG8=" assert urlsafe_b64encode(b"\xfb\xff") == "-_8=" match b64decode_lenient("aG Vs\nbG8="): @@ -50,16 +51,8 @@ def main() -> None: match b64decode("a=AA"): Ok(_) => assert false, "invalid-padding base64 unexpectedly decoded" Err(err) => assert err.kind == "invalid_padding" -"#, - ) -} -#[test] -fn base32_vectors_and_lenient_decode() -> Result<(), Box> { - run_module_case( - "crates/incan_stdlib/stdlib/encoding/base32.incn", - r#" -def main() -> None: +def check_base32() -> None: assert b32encode(b"foo") == "MZXW6===" assert b32hexencode(b"foo") == "CPNMU===" match b32decode_lenient("mz xw6==="): @@ -71,16 +64,8 @@ def main() -> None: match b32decode("MZ=XW6=="): Ok(_) => assert false, "misplaced-padding base32 unexpectedly decoded" Err(err) => assert err.kind == "invalid_padding" -"#, - ) -} -#[test] -fn base58_vectors_and_lenient_decode() -> Result<(), Box> { - run_module_case( - "crates/incan_stdlib/stdlib/encoding/base58.incn", - r#" -def main() -> None: +def check_base58() -> None: assert b58encode(b"hello world") == "StV1DL6CwTryKyV" assert b58encode(b"\x00\x00") == "11" match b58decode_lenient(" StV1DL6CwTryKyV\n"): @@ -89,16 +74,8 @@ def main() -> None: match b58decode("0"): Ok(_) => assert false, "invalid base58 unexpectedly decoded" Err(err) => assert err.kind == "invalid_character" -"#, - ) -} -#[test] -fn base85_vectors_and_lenient_decode() -> Result<(), Box> { - run_module_case( - "crates/incan_stdlib/stdlib/encoding/base85.incn", - r#" -def main() -> None: +def check_base85() -> None: assert a85encode(b"\x00\x00\x00\x00") == "z" match b85decode(b85encode(b"hello")): Ok(data) => assert data == b"hello" @@ -118,20 +95,12 @@ def main() -> None: match b85decode("\t"): Ok(_) => assert false, "invalid-character base85 unexpectedly decoded" Err(err) => assert err.kind == "invalid_character" -"#, - ) -} -#[test] -fn bech32_vectors_and_invalid_cases() -> Result<(), Box> { - run_module_case( - "crates/incan_stdlib/stdlib/encoding/bech32.incn", - r#" -def main() -> None: +def check_bech32() -> None: match bech32_encode("a", []): Ok(text) => assert text == "a12uel5l" Err(err) => assert false, err.detail - match decode("A12UEL5L"): + match bech32_decode_any("A12UEL5L"): Ok(decoded) => assert decoded.hrp == "a" and len(decoded.data) == 0 and decoded.variant == Bech32Variant.Bech32 Err(err) => assert false, err.detail match bech32m_encode("a", []): @@ -140,6 +109,13 @@ def main() -> None: match bech32_decode("a1lqfn3a"): Ok(_) => assert false, "bech32 accepted a bech32m checksum" Err(err) => assert err.kind == "invalid_checksum" + +def main() -> None: + check_base64() + check_base32() + check_base58() + check_base85() + check_bech32() "#, ) } diff --git a/tests/vocab_guardrails.rs b/tests/vocab_guardrails.rs index 60cdfa67c..06cceba52 100644 --- a/tests/vocab_guardrails.rs +++ b/tests/vocab_guardrails.rs @@ -4,6 +4,9 @@ use std::path::{Path, PathBuf}; use incan_core::lang::derives; use incan_core::lang::types::collections; +use serde::Deserialize; + +const SEMANTIC_STRING_AUDIT_PATH: &str = "tests/fixtures/vocab_guardrails/semantic_string_audit.json"; /// Guardrail against reintroducing stringly-typed vocabulary checks. /// @@ -42,10 +45,184 @@ fn no_new_stringly_vocab_checks_in_rust_sources() { } } +#[derive(Debug)] +struct AuditedSemanticStringFile { + path: String, + category: String, + expected_count: usize, + expected_fingerprint: u64, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct SemanticStringAudit { + files: Vec, +} + +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields)] +struct RawAuditedSemanticStringFile { + path: String, + category: String, + expected_count: usize, + expected_fingerprint: String, +} + +/// Guardrail for semantic string comparisons that remain in high-risk compiler paths. +/// +/// A semantic string comparison is not automatically wrong. Some strings are source names, manifest keys, Rust display +/// fragments, or quarantined metadata-free compatibility policy. The point of this test is that these comparisons must +/// be visible and classified instead of silently growing in typechecking, lowering, emission, dependency resolution, or +/// Rust inspection. +#[test] +fn semantic_string_checks_are_classified() { + let root = repo_root(); + let audit_entries = audited_semantic_string_files(&root); + let scan_files = semantic_string_scan_files(&root); + let scanned_paths: BTreeSet = scan_files.iter().map(|path| rel_path(&root, path)).collect(); + let mut offenders = Vec::new(); + + for path in &scan_files { + let sites = semantic_string_sites(path); + if sites.is_empty() { + continue; + } + let rel = rel_path(&root, path); + let actual_count = sites.len(); + let actual_fingerprint = fingerprint_sites(&sites); + match audit_entries.iter().find(|entry| entry.path == rel) { + Some(entry) if entry.expected_count == actual_count && entry.expected_fingerprint == actual_fingerprint => { + } + Some(entry) => offenders.push(format!( + "{} changed in `{}`: expected {} sites/{:016x}, found {} sites/{:016x}", + entry.category, rel, entry.expected_count, entry.expected_fingerprint, actual_count, actual_fingerprint + )), + None => offenders.push(format!( + "unclassified semantic string checks in `{rel}`: {} sites/{actual_fingerprint:016x}", + actual_count + )), + } + } + + let mut audited_paths: BTreeSet<&str> = BTreeSet::new(); + let mut previous_audited_path: Option<&str> = None; + for entry in &audit_entries { + if let Some(previous) = previous_audited_path + && previous > entry.path.as_str() + { + offenders.push(format!( + "semantic string audit paths are not sorted: `{previous}` appears before `{}`", + entry.path + )); + } + previous_audited_path = Some(entry.path.as_str()); + if !audited_paths.insert(entry.path.as_str()) { + offenders.push(format!("duplicate semantic string audit entry: `{}`", entry.path)); + } + let path = root.join(&entry.path); + if !path.exists() { + offenders.push(format!( + "audited semantic string file no longer exists: `{}` ({})", + entry.path, entry.category + )); + } else if !scanned_paths.contains(&entry.path) { + offenders.push(format!( + "audited semantic string file is outside scanned roots: `{}` ({})", + entry.path, entry.category + )); + } + } + + if !offenders.is_empty() { + let mut msg = String::new(); + msg.push_str( + "Semantic string checks changed. Move behavior behind a registry when possible; otherwise classify the file in the semantic string audit fixture.\n\n", + ); + for offender in offenders { + msg.push_str("- "); + msg.push_str(&offender); + msg.push('\n'); + } + msg.push_str("\nCurrent scanned sites:\n"); + for path in &scan_files { + let sites = semantic_string_sites(path); + if sites.is_empty() { + continue; + } + let rel = rel_path(&root, path); + msg.push_str(&format!( + "\n{rel} ({} sites/{:016x})\n", + sites.len(), + fingerprint_sites(&sites) + )); + for site in sites { + msg.push_str(&format!(" {}\n", site.trim())); + } + } + panic!("{msg}"); + } +} + fn repo_root() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")) } +fn rel_path(root: &Path, path: &Path) -> String { + path.strip_prefix(root) + .unwrap_or(path) + .to_string_lossy() + .replace('\\', "/") +} + +fn audited_semantic_string_files(root: &Path) -> Vec { + let audit_path = root.join(SEMANTIC_STRING_AUDIT_PATH); + let contents = fs::read_to_string(&audit_path) + .unwrap_or_else(|err| panic!("failed to read semantic string audit `{}`: {err}", audit_path.display())); + let audit: SemanticStringAudit = serde_json::from_str(&contents).unwrap_or_else(|err| { + panic!( + "failed to parse semantic string audit `{}`: {err}", + audit_path.display() + ) + }); + + if audit.files.is_empty() { + panic!( + "semantic string audit `{}` must classify at least one file", + audit_path.display() + ); + } + + audit + .files + .into_iter() + .map(|entry| { + let expected_fingerprint = + parse_expected_fingerprint(&audit_path, &entry.path, &entry.expected_fingerprint); + AuditedSemanticStringFile { + path: entry.path, + category: entry.category, + expected_count: entry.expected_count, + expected_fingerprint, + } + }) + .collect() +} + +fn parse_expected_fingerprint(audit_path: &Path, entry_path: &str, value: &str) -> u64 { + let hex = value.strip_prefix("0x").unwrap_or_else(|| { + panic!( + "semantic string audit `{}` entry `{entry_path}` has non-hex expected_fingerprint `{value}`", + audit_path.display() + ) + }); + u64::from_str_radix(hex, 16).unwrap_or_else(|err| { + panic!( + "semantic string audit `{}` entry `{entry_path}` has invalid expected_fingerprint `{value}`: {err}", + audit_path.display() + ) + }) +} + fn tier_a_spellings() -> Vec<&'static str> { // Tier A: high-signal, drift-prone vocabulary. // - Generic bases / builtin collection type names (and aliases) @@ -131,3 +308,211 @@ fn is_suspicious_line(line: &str, spellings: &[&'static str]) -> bool { false } + +fn semantic_string_scan_files(root: &Path) -> Vec { + const ROOTS: &[&str] = &[ + "crates/incan_core/src/interop", + "crates/rust_inspect/src", + "src/backend/ir", + "src/cli/commands/common.rs", + "src/dependency_resolver.rs", + "src/frontend/testing_markers.rs", + "src/frontend/typechecker", + ]; + + let mut files = Vec::new(); + for root_path in ROOTS { + collect_rust_files(&root.join(root_path), &mut files); + } + files.sort(); + files.dedup(); + files +} + +fn collect_rust_files(path: &Path, files: &mut Vec) { + if path.is_file() { + if is_semantic_string_scan_file(path) { + files.push(path.to_path_buf()); + } + return; + } + + let Ok(entries) = fs::read_dir(path) else { + return; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + collect_rust_files(&path, files); + } else if is_semantic_string_scan_file(&path) { + files.push(path); + } + } +} + +fn is_semantic_string_scan_file(path: &Path) -> bool { + path.extension().and_then(|ext| ext.to_str()) == Some("rs") + && path.file_name().and_then(|file| file.to_str()) != Some("tests.rs") + && !path + .components() + .any(|component| component.as_os_str().to_str() == Some("tests")) +} + +fn semantic_string_sites(path: &Path) -> Vec { + let Ok(contents) = fs::read_to_string(path) else { + return Vec::new(); + }; + let mut sites = Vec::new(); + let mut brace_depth = 0usize; + let mut pending_cfg_test = false; + let mut skip_until_depth: Option = None; + + for line in contents.lines() { + let code = strip_line_comment(line).trim(); + if let Some(target_depth) = skip_until_depth { + brace_depth = update_brace_depth(brace_depth, code); + if brace_depth <= target_depth { + skip_until_depth = None; + } + continue; + } + + if code.starts_with("#[cfg(test)]") { + pending_cfg_test = true; + brace_depth = update_brace_depth(brace_depth, code); + continue; + } + if pending_cfg_test && code.contains("mod tests") && code.contains('{') { + let target_depth = brace_depth; + brace_depth = update_brace_depth(brace_depth, code); + if brace_depth > target_depth { + skip_until_depth = Some(target_depth); + } + pending_cfg_test = false; + continue; + } + if pending_cfg_test && !code.starts_with("#[") && !code.is_empty() { + pending_cfg_test = false; + } + + if semantic_string_line(code) { + sites.push(code.to_string()); + } + brace_depth = update_brace_depth(brace_depth, code); + } + + sites +} + +fn update_brace_depth(current: usize, code: &str) -> usize { + let mut depth = current; + let mut in_string = false; + let mut escaped = false; + for byte in code.bytes() { + if in_string { + if escaped { + escaped = false; + } else if byte == b'\\' { + escaped = true; + } else if byte == b'"' { + in_string = false; + } + continue; + } + match byte { + b'"' => in_string = true, + b'{' => depth = depth.saturating_add(1), + b'}' => depth = depth.saturating_sub(1), + _ => {} + } + } + depth +} + +fn strip_line_comment(line: &str) -> &str { + let mut in_string = false; + let mut escaped = false; + let bytes = line.as_bytes(); + let mut idx = 0usize; + while idx + 1 < bytes.len() { + let byte = bytes[idx]; + if in_string { + if escaped { + escaped = false; + } else if byte == b'\\' { + escaped = true; + } else if byte == b'"' { + in_string = false; + } + idx += 1; + continue; + } + if byte == b'"' { + in_string = true; + } else if byte == b'/' && bytes[idx + 1] == b'/' { + return &line[..idx]; + } + idx += 1; + } + line +} + +fn semantic_string_line(code: &str) -> bool { + if code.is_empty() || !code.contains('"') { + return false; + } + if code.starts_with("#[") + || code.starts_with("assert!") + || code.starts_with("assert_eq!") + || code.starts_with("assert_ne!") + || code.starts_with("panic!") + || code.starts_with("format!") + || code.starts_with("write!") + || code.starts_with("writeln!") + { + return false; + } + + line_has_string_comparison(code) + || line_has_string_matches_macro(code) + || line_has_string_match_arm(code) + || line_has_semantic_string_table(code) +} + +fn line_has_string_comparison(code: &str) -> bool { + code.contains("== \"") + || code.contains("!= \"") + || code.contains("== &\"") + || code.contains("!= &\"") + || code.contains(".as_deref() == Some(\"") + || code.contains(".as_deref() != Some(\"") + || code.contains("== Some(\"") + || code.contains("!= Some(\"") +} + +fn line_has_string_matches_macro(code: &str) -> bool { + code.contains("matches!(") && code.contains('"') +} + +fn line_has_string_match_arm(code: &str) -> bool { + let Some(arrow_idx) = code.find("=>") else { + return false; + }; + let before_arrow = code[..arrow_idx].trim_start(); + before_arrow.starts_with('"') || before_arrow.starts_with("| \"") || before_arrow.starts_with("(\"") +} + +fn line_has_semantic_string_table(code: &str) -> bool { + code.contains("methods: &[") || code.contains("expected: &[") || code.contains("features: &[") +} + +fn fingerprint_sites(sites: &[String]) -> u64 { + let mut hash = 0xcbf29ce484222325u64; + for site in sites { + for byte in site.as_bytes().iter().chain(std::iter::once(&b'\n')) { + hash ^= u64::from(*byte); + hash = hash.wrapping_mul(0x100000001b3); + } + } + hash +} diff --git a/workspaces/docs-site/docs/RFCs/034_incan_pub_registry.md b/workspaces/docs-site/docs/RFCs/034_incan_pub_registry.md index efa76f6e8..d4b75e5f4 100644 --- a/workspaces/docs-site/docs/RFCs/034_incan_pub_registry.md +++ b/workspaces/docs-site/docs/RFCs/034_incan_pub_registry.md @@ -13,6 +13,8 @@ Define the `incan.pub` package registry: the protocols, guarantees, and CLI commands that allow Incan library authors to publish packages and consumers to resolve them. The registry must be EU-hosted, integrity-verified, signature-aware, and operationally cheap enough to run with predictable capped spend. Exact vendor choice and launch-era cost numbers are implementation details, not the core contract. +This Draft was originally written against RFC 031's generated-Rust library artifact shape. The package format and resolution model below are now amended to align with the backend replacement direction: generated Rust may remain an internal/debug artifact, but it is not the public package compatibility path. The registry stores Incan package artifacts with semantic manifests, ABI/package metadata, and optional backend artifacts; consumers resolve Incan semantics first, not downloaded generated Rust source. + ## Constraints Two non-negotiable requirements drive every decision in this RFC: @@ -139,21 +141,22 @@ This is a static site generated from the index — no dynamic server needed for ### The package format -A `.crate` file is a gzipped tarball containing the Rust crate output from `incan build --lib` plus the `.incnlib` type manifest: +An Incan package artifact is a compressed archive, conventionally `.incanpkg`, containing package metadata, semantic manifests, ABI/package metadata, and optional emitted artifacts: ```text -mylib-0.1.0.crate (tar.gz): +mylib-0.1.0.incanpkg (tar.gz): └── mylib-0.1.0/ - ├── Cargo.toml # Generated Rust crate metadata - ├── src/ - │ ├── lib.rs # Generated Rust source - │ └── widgets.rs - └── .incnlib # Type manifest (JSON, from RFC 031) + ├── incan-package.json # Package identity, dependencies, ABI/schema versions + ├── .incnlib # Checked type/API manifest + ├── semantic/ # Optional semantic package fragments + ├── abi/ # Optional Rust-facing ABI/package metadata + ├── src/ # Optional source snapshot, when publishing policy allows it + └── artifacts/ # Optional target artifacts and inspection reports ``` -The `.incnlib` file is invisible to Cargo (which ignores unknown files in the tarball). The `incan` CLI extracts it for typechecking; `cargo build` only sees the Rust source. +Generated Rust source is not required to be present and must not be the public compatibility contract. If an implementation includes generated Rust for inspection, debugging, or migration, that output is an artifact with provenance metadata, not the semantic source of truth for package consumers. -This is a single artifact — the type manifest and compiled Rust source are never stored or transferred separately. This simplifies every part of the pipeline: publish uploads one file, download retrieves one file, cache stores one file. +This is still one immutable package artifact for the registry: publish uploads one archive, download retrieves one archive, cache stores one archive, and checksums/signatures cover the archive as a whole. The compiler and backend decide how to consume package semantics and emit target-specific code for the current build. ### Index format @@ -174,9 +177,10 @@ index/my/li/mylib |---|---|---| | `name` | string | Package name | | `vers` | string | SemVer version | -| `cksum` | string | SHA256 of the `.crate` tarball (prefixed with `sha256:`) | +| `cksum` | string | SHA256 of the package archive (prefixed with `sha256:`) | | `deps` | array | Incan library dependencies (`name` + `req` version range) | -| `rust_deps` | array | Rust crate dependencies (merged into consumer's Cargo.toml) | +| `rust_deps` | array | Rust crate dependencies required by package backend/ABI metadata, resolved by the compiler backend rather than blindly merged into user-authored manifests | +| `artifact_kind` | string | Package artifact format, such as `incanpkg` | | `incan_version` | string | Minimum compiler version required | | `yanked` | bool | If true, existing lockfiles still resolve but new resolves skip | | `publisher` | string | Publisher identity (username) | @@ -207,7 +211,7 @@ Headers: X-Signature: MEUC... (base64, optional in Phase 1) X-Certificate: MIIB... (base64, optional in Phase 1) -Body: .crate tarball (binary) +Body: Incan package archive (binary) ``` **Server-side validation:** @@ -217,11 +221,13 @@ Body: .crate tarball (binary) 3. Verify `(name, version)` does not already exist → 409 Conflict 4. Verify `X-Checksum` matches SHA256 of request body 5. If signature provided: verify Sigstore signature is valid, signer matches publisher -6. Extract `.incnlib` from tarball → verify it parses (basic structural validation) -7. Store `.crate` in object storage: `crates//.crate` -8. Store signature artifacts: `crates//.crate.sig`, `.cert` -9. Update index: append version line to `index//` -10. Invalidate CDN cache for the index entry 11. Return 200 +6. Extract `incan-package.json` and `.incnlib` from the archive and verify they parse +7. Reject archives that require generated Rust source as the package compatibility path +8. Store package archive in object storage: `packages//.incanpkg` +9. Store signature artifacts: `packages//.incanpkg.sig`, `.cert` +10. Update index: append version line to `index//` +11. Invalidate CDN cache for the index entry +12. Return 200 **Response:** `{ "published": "mylib", "version": "0.1.0" }` @@ -233,15 +239,15 @@ Headers: Body: { "name": "mylib", "version": "0.1.0" } ``` -Sets `yanked: true` in the index entry. Does not delete the `.crate` file (existing lockfiles and builds that reference this exact version still work). +Sets `yanked: true` in the index entry. Does not delete the package archive (existing lockfiles and builds that reference this exact version still work). #### `GET /index//` Returns the JSON-lines index file for the named package. Served from object storage, cached at CDN edge. -#### `GET /crates//.crate` +#### `GET /packages//.incanpkg` -Returns the `.crate` tarball. Served from object storage, cached at CDN edge. Immutable forever — cache headers set to maximum TTL. +Returns the package archive. Served from object storage, cached at CDN edge. Immutable forever, with cache headers set to maximum TTL. ### Authentication @@ -270,22 +276,22 @@ $ incan login ### Package signing with Sigstore -Every `incan publish` signs the `.crate` tarball using [Sigstore](https://sigstore.dev) keyless signing: +Every `incan publish` signs the package archive using [Sigstore](https://sigstore.dev) keyless signing: **Publish side:** 1. `incan publish` initiates an OIDC flow (opens browser → GitHub/GitLab/Google login) 2. Sigstore's Fulcio CA issues a short-lived signing certificate tied to the OIDC identity -3. The `.crate` file's SHA256 digest is signed with the ephemeral private key +3. The package archive's SHA256 digest is signed with the ephemeral private key 4. The signature + certificate + checksum are recorded in Sigstore's Rekor transparency log -5. The signature and certificate are sent to the registry alongside the `.crate` +5. The signature and certificate are sent to the registry alongside the package archive **Verification side (`incan build`):** -1. Download `.crate` + `.sig` + `.cert` from registry -2. Verify SHA256 of `.crate` matches the index checksum +1. Download package archive + `.sig` + `.cert` from registry +2. Verify SHA256 of the archive matches the index checksum 3. Verify the certificate was issued by Sigstore Fulcio CA -4. Verify the signature matches the `.crate` digest +4. Verify the signature matches the archive digest 5. Verify the signer identity in the certificate matches the `publisher` field in the index 6. Verify the signature is recorded in Sigstore Rekor (transparency log lookup) @@ -325,12 +331,14 @@ Resolution: 2. For each registry dep: `GET https://incan.pub/index//` 3. Parse JSON lines, filter by version requirement, select newest matching non-yanked version 4. Check local cache `~/.incan/libs/-/` — if cached and checksum matches, skip download -5. `GET https://incan.pub/crates//.crate` +5. `GET https://incan.pub/packages//.incanpkg` 6. Verify SHA256 checksum matches index entry 7. Verify Sigstore signature (if present; warn if absent) 8. Extract to `~/.incan/libs/-/` -9. Load `.incnlib` into typechecker symbol table -10. Wire Rust crate as path dependency in generated `Cargo.toml` +9. Load `.incnlib`, package metadata, and ABI/semantic facts into the compiler package database +10. Let the backend consume those package facts and emit the target build artifacts + +The resolver must not wire downloaded generated Rust source into generated `Cargo.toml` as the package compatibility path. Rust-facing consumption should go through the ABI/Cargo-native package direction rather than treating generated Rust internals as public API. **Lockfile (`incan.lock`):** on first resolution, write resolved versions + checksums to `incan.lock`. Subsequent builds use the lockfile for reproducibility. `incan update` re-resolves. @@ -342,7 +350,7 @@ Resolution: | `incan remove ` | Remove a dependency from `incan.toml` | | `incan update` | Re-resolve all dependencies and update `incan.lock` | | `incan login` | Authenticate with `incan.pub`, save token to `~/.incan/credentials` | -| `incan publish` | Build library, package `.crate`, sign, upload to registry | +| `incan publish` | Build library, package `.incanpkg`, sign, upload to registry | | `incan yank ` | Mark a version as yanked (still downloadable but skipped in new resolves) | | `incan search ` | Search the registry index (client-side text search over cached index) | | `incan owner add ` | Add a co-owner for a package | @@ -432,10 +440,10 @@ The registry service should talk to object storage via an S3-compatible API or e Kellnr is a self-hosted Rust crate registry that implements the Cargo registry protocol. It was considered and rejected because: -- It only speaks the Cargo registry protocol — no awareness of `.incnlib` manifests +- It only speaks the Cargo registry protocol and has no awareness of Incan package manifests, semantic metadata, or ABI metadata - Requires a persistent server (no scale-to-zero) - Written in Rust, not Incan (misses the dogfooding opportunity) -- The `.incnlib`-in-`.crate` trick makes Cargo protocol compatibility free anyway — any tool that can download a `.crate` gets both the Rust source and the type manifest +- Treating generated Rust as a Cargo package artifact would recreate the public-compatibility path the backend direction is moving away from ## Reference service implementation (informative) @@ -449,9 +457,9 @@ The important design constraint is portability: ## Interaction with existing features -- **RFC 031 (library system):** This RFC builds directly on RFC 031. The `.incnlib` manifest format, `pub::` import syntax, and `incan build --lib` command are defined there. This RFC adds the distribution layer on top. -- **RFC 027 (incan-vocab):** Library soft keyword declarations are serialized into the `.incnlib` manifest during `incan build --lib` and included in the `.crate` tarball. The registry is unaware of soft keywords — it just stores and serves packages. -- **`rust::` imports (RFC 005):** `pub::` registry imports and `rust::` Rust crate imports coexist. A package's Rust dependencies (from its generated `Cargo.toml`) are listed in the index entry's `rust_deps` field. +- **RFC 031 (library system):** This RFC builds on RFC 031's `.incnlib` manifest format, `pub::` import syntax, and `incan build --lib` command, but supersedes any assumption that generated Rust source is the registry package contract. +- **RFC 027 (incan-vocab):** Library soft keyword declarations are serialized into checked package metadata during `incan build --lib` and included in the package archive. The registry is unaware of soft keywords; it stores and serves packages. +- **`rust::` imports (RFC 005):** `pub::` registry imports and `rust::` Rust crate imports coexist. A package's Rust dependencies may appear in package metadata, but the compiler backend owns how they are linked into the target build. ## Alternatives considered diff --git a/workspaces/docs-site/docs/RFCs/066_std_http.md b/workspaces/docs-site/docs/RFCs/066_std_http.md index e0779ba8f..e581a532d 100644 --- a/workspaces/docs-site/docs/RFCs/066_std_http.md +++ b/workspaces/docs-site/docs/RFCs/066_std_http.md @@ -10,6 +10,8 @@ - RFC 051 (`JsonValue` for `std.json`) - RFC 055 (`std.fs` path-centric filesystem APIs) - RFC 063 (`std.process` process spawning and command execution) + - RFC 078 (tool execution and typed workflow actions) + - RFC 103 (`std.secrets` secret strings and bytes) - **Issue:** https://github.com/dannys-code-corner/incan/issues/84 - **RFC PR:** — - **Written against:** v0.2 @@ -17,16 +19,18 @@ ## Summary -This RFC proposes `std.http` as Incan's standard library module for explicit HTTP client work. The module standardizes a request or response model, one-shot and client-based request APIs, timeout and retry policy, structured errors, and JSON convenience surfaces so ordinary programs, tools, and automation workflows do not need to fall through to `rust::reqwest`-shaped APIs or ad hoc wrappers. +This RFC proposes `std.http` as Incan's standard library module for explicit HTTP client work. The module standardizes a request or response model, one-shot and client-based request APIs, client lifecycle, protocol negotiation, timeout and retry policy, structured errors, and JSON convenience surfaces so ordinary programs, tools, and automation workflows do not need to fall through to `rust::reqwest`-shaped APIs or ad hoc wrappers. ## Core model -Read this RFC as one foundation plus three mechanisms: +Read this RFC as one foundation plus five mechanisms: 1. **Foundation:** HTTP is a general-purpose stdlib capability, not a CI-only or framework-only helper surface. 2. **Mechanism A:** `std.http` provides explicit `Request`, `Response`, `Body`, `Method`, and `HttpError` types with predictable behavior and no panic-driven network contract. 3. **Mechanism B:** the module supports both one-shot convenience helpers and a reusable `Client` surface so simple scripts and heavier integrations share one coherent model. -4. **Mechanism C:** JSON, timeout, redirect, and retry behavior remain explicit policy surfaces rather than ambient magic. +4. **Mechanism C:** client lifecycle and pooling are explicit enough that repeated calls do not depend on hidden global connection state. +5. **Mechanism D:** JSON, timeout, redirect, and retry behavior remain explicit policy surfaces rather than ambient magic. +6. **Mechanism E:** HTTP protocol negotiation, streaming, and test transports remain inspectable seams instead of backend-specific escape hatches. ## Motivation @@ -36,6 +40,55 @@ This matters for more than ergonomics. HTTP boundaries are policy-heavy: timeout `std.http` should therefore do for network requests what `std.fs`, `std.process`, and the newer stdlib RFCs are doing in their domains: define an Incan-first contract while still allowing the runtime to map onto Rust-native implementations underneath. +## HTTP client prior art + +### Requests baseline + +Python's `requests` is useful as the ergonomic baseline. Its quickstart frames ordinary HTTP verbs as obvious one-line calls, while still returning a response object the caller can inspect. Incan should keep that floor: a health check, webhook call, artifact fetch, or small API client should not require building a full framework object graph. + +Source: [Requests quickstart](https://docs.python-requests.org/en/latest/user/quickstart/). + +The Incan lesson is: + +- method-specific helpers such as `get`, `post`, `put`, and `delete` are worth keeping +- helpers should return the same response model as explicit requests +- simple does not mean ambient: timeouts, errors, redaction, and policy still need defined behavior +- the public API should be obvious before it is powerful + +### HTTPX lessons + +HTTPX is useful prior art because it modernizes the `requests` shape without reducing the design to convenience helpers. Its documentation presents a fully featured client with sync and async APIs, HTTP/1.1 and HTTP/2 support, strict timeouts, async clients for async frameworks, and opt-in HTTP/2 with response-level protocol visibility. + +Sources: [HTTPX introduction](https://www.python-httpx.org/), [HTTPX async support](https://www.python-httpx.org/async/), and [HTTPX HTTP/2 support](https://www.python-httpx.org/http2/). + +The Incan lesson is not to copy Python's split between `Client` and `AsyncClient` literally. The useful design pressure is: + +- a reusable client is a real resource, not just a namespace for functions +- connection pooling and cleanup should be visible in the API contract +- one-shot helpers are useful, but repeated requests should have an obvious client-owned path +- timeout policy should be present by default and refinable later into connect/read/write/overall timeout fields +- HTTP/2 should be an explicit protocol policy, not an accidental backend behavior +- responses should expose the negotiated protocol version +- streaming and test transports should fit the same `Request` / `Response` / `HttpError` vocabulary + +Incan should go further than HTTPX where the language gives it leverage: typed errors instead of exception families, model-aware JSON decoding, capability-gated network access, and policy-visible remote data flow for tools, CI, and AI-backed actions. + +### Koheesio lessons + +Koheesio is useful prior art because it treats HTTP as a pipeline step concern, not only as an ad hoc client call. Its HTTP step surface includes method-specific steps, a shared request configuration shape, timeout options, retry behavior, response outputs such as raw payload, JSON payload, and status code, paginated HTTP GET support, and explicit masking for sensitive authorization headers. Its async HTTP step also makes session, retry, and connector state visible. + +Sources: [Koheesio HTTP steps](https://engineering.nike.com/koheesio/0.10.0/api_reference/steps/http.html) and [Koheesio async HTTP steps](https://engineering.nike.com/koheesio/0.10.0/api_reference/asyncio/http.html). + +The Incan lesson is: + +- `std.http` should be general-purpose, but its request and response types must compose cleanly with step, pipeline, and typed-action systems +- retry, timeout, pagination, and authorization are operational concerns, not just transport knobs +- response projections should be stable enough for workflow outputs, logs, quality checks, and tests +- sensitive header handling belongs in the core design, not only in logging docs +- async execution should make session and connector ownership visible without forcing backend-specific types into user code + +Incan should not copy Koheesio's Python/Pydantic runtime boundary literally. The stdlib contract should preserve the step-friendly shape while using `Result[..., HttpError]`, typed request/response models, compile-time metadata, and Rust-native execution underneath. + ## Goals - Provide a first-class `std.http` module for client-side HTTP work. @@ -44,8 +97,13 @@ This matters for more than ergonomics. HTTP boundaries are policy-heavy: timeout - Define a structured `HttpError` model so network failures, status failures, timeout failures, decoding failures, and policy failures are distinguishable. - Provide JSON convenience helpers that compose cleanly with RFC 051 `JsonValue`. - Support both one-shot request helpers and a reusable `Client` surface. +- Make `Client` lifecycle, cleanup, and reuse explicit enough to support connection pooling without hidden globals. +- Make negotiated HTTP protocol information visible on responses, while avoiding a v1 requirement that every backend support HTTP/2. +- Keep request and response models structured enough to compose with typed workflow actions, pipeline steps, logs, tests, and generated reports. - Make retry behavior explicit and policy-shaped rather than automatic and invisible. +- Leave room for streaming bodies and test transports without leaking backend-specific transport types. - Require safe default treatment of sensitive headers in diagnostics and debug-facing representations. +- Accept secret value types for authentication and header-building APIs so callers do not need to reveal tokens into plain strings before sending requests. ## Non-Goals @@ -54,6 +112,7 @@ This matters for more than ergonomics. HTTP boundaries are policy-heavy: timeout - Making HTTP a language intrinsic or keyword surface. - Introducing a GitHub- or cloud-specific SDK into the standard library. - Standardizing cookies, OAuth flows, multipart forms, WebSockets, or HTTP/3-specific behavior in the first version. +- Requiring HTTP/2 support from every v1 implementation. ## Guide-level explanation @@ -114,6 +173,27 @@ items = response.json()? This does not change the basic model. It only moves repeated policy into one reusable value. +### Client lifecycle and pooling + +A `Client` should be treated as a resource that owns transport state such as connection pools, default headers, timeout policy, retry policy, redirect policy, and protocol preferences. The exact cleanup spelling is left to the implementation, but the API must make deterministic cleanup possible. + +One-shot helpers are still valuable for scripts and probes. Repeated calls, service-to-service integrations, crawlers, SDKs, and long-running tools should have an obvious client path so code does not create a fresh transport stack in a hot loop. + +### Protocol negotiation + +HTTP/2 support should be explicit without making it mandatory for all implementations. A client or request should be able to declare a protocol policy: + +```incan +from std.http import Client, Protocol + +client = Client(protocol=Protocol.Http2Preferred) +response = client.get("https://api.example.com/items")? + +println(response.protocol) +``` + +The exact names can change, but the shape should support "use the backend default", "HTTP/1 only", "prefer HTTP/2", and "require HTTP/2." If HTTP/2 is required and the implementation cannot provide it, the result should be a structured `HttpError`, not a silent downgrade. + ### Status handling should stay explicit The response model should not hide status behavior behind panics. Users should opt into strict status expectations: @@ -145,6 +225,8 @@ println(request) should not casually dump bearer tokens or secrets into logs. +When the caller uses `SecretStr` or `SecretBytes` from RFC 103, redaction should come from the value type as well as from conservative header-name rules. A header value derived from a secret wrapper must remain redacted even if the header name is custom. + ## Reference-level explanation ### Module surface @@ -158,6 +240,7 @@ should not casually dump bearer tokens or secrets into logs. - `StatusCode` - `HttpError` - `Client` +- protocol policy and negotiated protocol-version metadata, or equivalent types - one-shot request helpers or a functionally equivalent request entry surface - explicit retry-policy types if retry behavior is part of the request contract @@ -174,6 +257,7 @@ A `Request` must carry: - body - timeout policy - redirect policy if separately configurable +- protocol policy if the caller needs to override the client default - retry policy when the caller opts into retries A request must be constructible without requiring a `Client`. @@ -183,10 +267,13 @@ A request must be constructible without requiring a `Client`. A `Response` must expose: - status code +- negotiated protocol version when available - response headers - body bytes - helpers for decoding text and JSON +The response model should also define stable, tool-friendly projections for common workflow outputs, such as status code, raw text or bytes, parsed JSON when requested, and redacted diagnostic summaries. These projections let pipeline steps, typed actions, tests, and reports use HTTP results without scraping backend-specific response objects. + A response must not silently panic on unsuccessful status codes. Status-based failure should remain explicit through helpers such as `require_success()` or equivalent APIs. ### Error model @@ -199,11 +286,23 @@ A response must not silently panic on unsuccessful status codes. Status-based fa - timeout failures - redirect-policy failures - TLS or transport failures +- unsupported or failed protocol negotiation - decode failures - explicit status-policy failures The module may include richer variants, but it must not collapse all failures into one undifferentiated string. +### Client lifecycle + +A `Client` owns reusable transport state. The contract must define: + +- how a client is closed or otherwise released +- whether operations after cleanup fail with a structured error +- which options are client defaults versus per-request overrides +- how one-shot helpers scope any temporary client state + +The API should make client reuse the natural path for repeated requests. One-shot helpers may internally create and dispose of clients, but the docs should not encourage creating new reusable clients inside tight loops. + ### Timeouts Timeouts must be first-class and explicit. The contract must define: @@ -214,6 +313,19 @@ Timeouts must be first-class and explicit. The contract must define: This RFC intentionally does not hardcode one exact default timeout yet; see unresolved questions. +Timeouts may start as one total request timeout, but the API should not block later support for distinct connect, read, write, and overall timeout fields. + +### Protocol negotiation + +The public contract should not assume that HTTP/1.1 is the only possible transport. It should standardize a small protocol-policy vocabulary, exact names pending: + +- backend default / automatic negotiation +- HTTP/1 only +- HTTP/2 preferred +- HTTP/2 required + +Implementations that do not support HTTP/2 may reject HTTP/2-preferred policies up front, or accept them and fall back to HTTP/1.x. HTTP/2-required policies must fail with a structured `HttpError` when the implementation, target, or peer cannot provide HTTP/2. If an implementation accepts a preferred policy and downgrades to HTTP/1.x, the `Response` must expose the protocol that was actually used. + ### Retries Retries must be opt-in and policy-shaped. A retry policy may cover: @@ -225,6 +337,12 @@ Retries must be opt-in and policy-shaped. A retry policy may cover: The module must not silently retry every request by default. +### Pagination and workflow composition + +The base `std.http` module does not need to standardize one pagination framework. It should, however, keep request construction, response decoding, and client reuse composable enough for libraries to build paginated fetchers, polling loops, and API-specific steps on top of the same primitives. + +Pipeline or workflow integrations should depend on `std.http` request/response models, not backend transport objects. A workflow action that fetches remote data should be able to report its URL policy, timeout, retry policy, status code, body shape, and redacted diagnostics through machine-readable action output. + ### JSON integration `Body.json(value)` or an equivalent API may accept `JsonValue` and, where later RFCs standardize model-oriented JSON encoding, other serializable values. @@ -235,7 +353,21 @@ The module must not silently retry every request by default. Implementations should redact sensitive header values such as `Authorization`, `Proxy-Authorization`, and similarly sensitive token-bearing headers in debug-facing request or response displays. -The public contract does not need to prescribe every redacted header name exhaustively in v1, but it must require that sensitive-header treatment is conservative and documented. +Header values constructed from RFC 103 `SecretStr` or `SecretBytes` must be treated as sensitive regardless of header name. Authentication helpers should accept secret value types directly so user code does not need to expose a token as a plain string before constructing a request. + +The public contract does not need to prescribe every redacted header name exhaustively in v1, but it must require that sensitive-header treatment is conservative and documented. Header-name heuristics are a fallback; value-level secret typing is the stronger contract when available. + +### Streaming and transports + +The first implementation does not need to support every streaming body shape, but the request and response model should leave room for: + +- streaming response bodies +- streaming request bodies +- explicit body size limits +- test transports that return synthetic responses without network access +- local application transports for testing `std.web` applications through the same client vocabulary + +Any transport abstraction must preserve `Request`, `Response`, `HttpError`, timeout, protocol, redaction, and policy semantics. Backend-specific transport handles must not become the public API. ## Design details @@ -248,6 +380,8 @@ This RFC does not require new language syntax. It is a namespaced stdlib surface The semantic center is explicit network behavior: - request creation is explicit +- client lifecycle is explicit +- protocol negotiation is visible - timeout policy is explicit - retry policy is explicit - status handling is explicit @@ -261,6 +395,8 @@ The module should not rely on hidden ambient globals for client state, retry beh - **RFC 055 (`std.fs`)**: file uploads or downloads may later compose with path or file surfaces, but this RFC does not require multipart or streaming file-transfer APIs. - **RFC 063 (`std.process`)**: HTTP should remain a direct network API, not a wrapper over shelling out to `curl`. - **RFC 037 (native web stdlib redesign)**: this RFC covers client-side HTTP. Server-side web contracts remain separate even if they eventually share types such as methods or status codes. +- **RFC 078 (tool execution and typed workflow actions)**: HTTP-capable tools and actions should be able to surface network access, protocol policy, and remote data flow through action metadata and policy checks. +- **RFC 103 (`std.secrets`)**: authentication helpers, header builders, diagnostics, retries, telemetry, and workflow output should preserve `SecretStr` and `SecretBytes` redaction semantics. ### Compatibility / migration @@ -276,30 +412,50 @@ This feature is additive. Existing Rust-interop HTTP wrappers remain valid, but - Rejected because real tooling and API clients need reusable policy and shared headers. - **Only `Client`, no one-shot helpers** - Rejected because it makes simple scripts too ceremonious. +- **A pipeline-specific HTTP step as the primary API** + - Rejected because HTTP is a general-purpose stdlib capability. Step and workflow libraries should compose over `std.http`; they should not own the base transport contract. +- **Separate public sync and async client models** + - Rejected for now because Incan should keep one conceptual client contract. Implementations may still provide blocking convenience helpers or async-only methods where the runtime requires them. +- **Mandatory HTTP/2 in v1** + - Rejected because the API should not block on backend coverage or target support. The important v1 contract is that protocol policy and negotiated protocol metadata have a place to live. +- **Hide protocol version entirely** + - Rejected because service-to-service clients, debugging, performance work, and policy checks sometimes need to know whether HTTP/1.x or HTTP/2 was actually used. +- **Expose backend transport types directly** + - Rejected because it would reintroduce the `rust::reqwest`-shaped leakage this RFC is trying to remove. ## Drawbacks - HTTP is a deceptively broad domain, and the API can sprawl if the module tries to cover every advanced transport concern immediately. - Timeout, retry, redirect, and status behavior need very careful wording or users will make conflicting assumptions. +- Protocol negotiation adds visible surface area before every implementation can support every protocol. +- Streaming and transport seams are easy to over-design if they are not tied to concrete tests and `std.web` integration cases. - Redaction rules and debug output need discipline or the module will create accidental secret leakage. ## Implementation architecture -*(Non-normative.)* A practical implementation likely uses a Rust-native HTTP stack underneath, but the public contract should remain request- and response-shaped. A sensible rollout would start with one-shot requests, explicit request objects, reusable clients, structured errors, timeouts, and `JsonValue` helpers before expanding into richer transport features such as multipart, streaming bodies, or cookie persistence. +*(Non-normative.)* A practical implementation likely uses a Rust-native HTTP stack underneath, but the public contract should remain request- and response-shaped. A sensible rollout would start with one-shot requests, explicit request objects, reusable clients, structured errors, timeouts, protocol metadata, and `JsonValue` helpers before expanding into richer transport features such as multipart, streaming bodies, cookie persistence, or HTTP/2 enforcement. ## Layers affected - **Stdlib / runtime**: must provide the request, response, method, body, client, and error surfaces promised by this RFC. - **Language surface**: the module and its helper types must be available as specified. -- **Execution handoff**: implementations must preserve timeout, retry, status, and decoding semantics without leaking backend-specific APIs as the public contract. +- **Execution handoff**: implementations must preserve timeout, retry, protocol, status, and decoding semantics without leaking backend-specific APIs as the public contract. - **Docs / tooling**: examples and documentation must standardize safe defaults, explicit status handling, and redaction expectations. ## Unresolved questions - Should `std.http` expose a default timeout at the module or client level, or should callers be required to choose one explicitly? +- Should timeout policy start as one total timeout, or should v1 expose connect/read/write/overall timeout fields immediately? - Should `Response.json()` standardize only `JsonValue` decoding in this RFC, or should typed model decoding be part of the base contract too? - Which redirect policy should be the default: follow a bounded number of redirects, or require explicit opt-in? - Should retry policies live on `Request`, `Client`, or both? +- Should protocol policy live on `Request`, `Client`, or both? +- Should HTTP/2 support be a v1 implementation feature, a v1 API shape with optional backend support, or a follow-up RFC? +- What is the minimum useful test transport: synthetic responses only, local `std.web` app transport, or a trait-like transport provider surface? +- What streaming body API is small enough for v1 while still compatible with large downloads and uploads later? +- Which response projections should be standardized for typed actions, pipeline steps, logs, and test assertions? +- Should pagination and polling helpers live in `std.http`, in workflow/step libraries, or in API-specific packages? - How much of cookie handling belongs in the initial contract versus a follow-up RFC? +- Which authentication helper shapes should accept `SecretStr` and `SecretBytes` directly in v1? diff --git a/workspaces/docs-site/docs/RFCs/079_incan_pub_artifact_graph.md b/workspaces/docs-site/docs/RFCs/079_incan_pub_artifact_graph.md index 7a0bc5dc9..927de3eef 100644 --- a/workspaces/docs-site/docs/RFCs/079_incan_pub_artifact_graph.md +++ b/workspaces/docs-site/docs/RFCs/079_incan_pub_artifact_graph.md @@ -207,6 +207,8 @@ The graph should represent advisories and yanking as relationships rather than o RFC 034 owns core package registry semantics. This RFC extends the registry's conceptual model from package versions to related artifact nodes and relationships. +This RFC inherits RFC 034's amended package artifact boundary: generated Rust source is not the public package compatibility path. Artifact graph nodes may describe generated implementation artifacts for inspection, provenance, compatibility reports, or migration, but package semantics must remain grounded in Incan manifests, semantic metadata, ABI/package metadata, and registry artifact relationships. + ### Relationship to RFC 074 and RFC 075 Template, starter, and capability descriptors are local tooling contracts. The graph can distribute and index them, but local lifecycle tooling owns rendering and mutation planning. diff --git a/workspaces/docs-site/docs/RFCs/092_interactive_runtime_stdlib_contracts.md b/workspaces/docs-site/docs/RFCs/092_interactive_runtime_stdlib_contracts.md index b259c14c0..3df822c9e 100644 --- a/workspaces/docs-site/docs/RFCs/092_interactive_runtime_stdlib_contracts.md +++ b/workspaces/docs-site/docs/RFCs/092_interactive_runtime_stdlib_contracts.md @@ -48,6 +48,7 @@ This RFC narrows the problem: Incan owns the contracts that make interactive run - Making WASM the default runtime posture. WASM may be one target capability, not the definition of interactive runtime support. - Defining native JSX, `html()` parsing, a component DSL, or a browser router in this RFC. - Defining GPU algorithms, shader language semantics, scene-graph APIs, physics engines, or rendering engines. +- Defining no-std/freestanding targets, kernel support, unsafe/layout controls, panic strategy, or allocator strategy. Runtime target manifests may inform that later work, but this RFC is not the freestanding/kernel RFC. - Replacing RFC 037 handler semantics. - Committing to a specific Rust web framework, JS framework, graphics crate, or bundler as the public contract. diff --git a/workspaces/docs-site/docs/RFCs/096_declaration_metadata_blocks.md b/workspaces/docs-site/docs/RFCs/096_declaration_metadata_blocks.md index 8764d4911..fa13a688b 100644 --- a/workspaces/docs-site/docs/RFCs/096_declaration_metadata_blocks.md +++ b/workspaces/docs-site/docs/RFCs/096_declaration_metadata_blocks.md @@ -12,7 +12,7 @@ - RFC 085 (field metadata and type-shaped constraints) - RFC 086 (schema descriptors and adapters) - RFC 091 (constrained integer newtype storage carriers) -- **Issue:** — +- **Issue:** https://github.com/dannys-code-corner/incan/issues/667 - **RFC PR:** — - **Written against:** v0.3 - **Shipped in:** — diff --git a/workspaces/docs-site/docs/RFCs/097_rust_hosted_incan_caller.md b/workspaces/docs-site/docs/RFCs/097_rust_hosted_incan_caller.md index b29df694b..f00f5bc33 100644 --- a/workspaces/docs-site/docs/RFCs/097_rust_hosted_incan_caller.md +++ b/workspaces/docs-site/docs/RFCs/097_rust_hosted_incan_caller.md @@ -20,49 +20,51 @@ ## Summary -This RFC defines a Rust-hosted Incan caller model: a native Rust application should be able to depend on an Incan-authored library through ordinary Cargo mechanics and call a curated, typed Rust-facing API without reverse-engineering generated code layout, manually wiring Incan runtime helpers, or treating every public Incan export as a stable Rust API. The model does not excuse poor generated Rust; the compiler must treat generated Rust as a first-class product surface. Incan should be a way for people and agents to author high-level Incan while producing great, idiomatic, fully-featured, opinionated Rust. The caller boundary is a higher-level host API shape built on top of that output, with generated adapters and a small support crate that own initialization, conversions, async/runtime policy, diagnostics, version checks, and panic/error containment. +This RFC defines a Rust-hosted Incan caller model: a native Rust application should be able to depend on an Incan-authored library through ordinary Cargo mechanics and call a curated, typed Rust-facing API without reverse-engineering compiler output, manually wiring Incan runtime helpers, or treating every public Incan export as a stable Rust API. + +This Draft is now framed around a Rust-facing caller ABI and Cargo-usable Incan package artifact. Generated Rust source may remain useful for inspection, debugging, migration, or an implementation backend, but it must not be the public package compatibility path. The caller boundary is the stable host API shape; it is backed by checked Incan metadata, ABI/package metadata, generated adapters where needed, and a small support crate that owns initialization, conversions, async/runtime policy, diagnostics, version checks, and panic/error containment. ## Core model 1. **Rust-hosted consumption is a first-class direction:** Incan already lets Incan code call Rust; this RFC defines the reverse direction where Rust code deliberately calls Incan-authored behavior. -2. **The generated Rust crate remains the compilation artifact:** RFC 031's generated library crate is still the concrete object Cargo builds and links. -3. **Generated Rust is a first-class product surface:** Rust-hosted consumption must not depend on a cleanup wrapper that hides bad emission. The emitted crate should be inspectable, idiomatic, documented, testable, debuggable, and useful to Rust users and tools. -4. **The caller boundary is the stable host-facing shape:** Rust consumers should target generated caller helpers and support traits that make calls feel natural from Rust while preserving Incan semantics. +2. **The Cargo-usable artifact is not generated Rust source as contract:** Rust hosts need a Cargo-native dependency shape, but the public compatibility promise is the caller ABI/package metadata, not compiler-emitted Rust internals. +3. **Implementation artifacts remain inspectable:** generated Rust, object code, IR snapshots, or other backend artifacts should be inspectable and debuggable where emitted, but they are not the host-facing semantic contract. +4. **The caller boundary is the stable host-facing shape:** Rust consumers should target caller helpers and support traits that make calls feel natural from Rust while preserving Incan semantics. 5. **The `pub` system should grow rather than be bypassed:** Rust-hosted exports should be modeled as a public export profile or facet, not as an unrelated side channel. 6. **Types cross through reusable helpers:** primitive values, models, newtypes, enums, `Result`, `Option`, collections, and Rust-backed types should cross through explicit, versioned conversion helpers that can also simplify emitter responsibilities. 7. **Runtime policy is explicit:** async execution, logger/telemetry hooks, host capabilities, panic handling, and initialization must be part of the caller contract rather than incidental generated code behavior. -8. **Cargo remains the host integration substrate:** Rust applications should use normal dependency declarations, build scripts, or generated package artifacts instead of a bespoke binary loader. +8. **Cargo remains the host integration substrate:** Rust applications should use normal dependency declarations, build scripts, or Cargo-usable package artifacts instead of a bespoke binary loader. ## Motivation Incan's current interop story is strong in one direction: Incan source imports Rust crates, wraps Rust types, and can implement Rust traits for Incan-owned types. That is necessary, but it does not answer the common embedding question: "how do I integrate Incan-generated code into my native Rust application code?" -That question exposes a deeper product direction. If Incan compiles to Rust, then generated Rust cannot be treated as a temporary compiler byproduct. It is one of the language's core deliverables. At minimum, Incan can become a disciplined way for people and agents to generate excellent Rust with strong opinions, complete runtime wiring, useful derives, reproducible packaging, diagnostics, tests, docs, and integration hooks included by default. +That question exposes a deeper product direction. Incan should produce Rust-native integration artifacts without making generated Rust source the package contract. Generated Rust can still be valuable as an implementation artifact and inspection surface, but the durable promise to Rust hosts should be an explicit caller ABI, metadata, support crate contract, and Cargo-native package shape. -RFC 031 already created the core artifact foundation: an Incan library can build a generated Rust crate plus a semantic manifest. That crate can technically be added as a Cargo path dependency today, and the compiler should make that generated crate good Rust. The missing product-level answer is the shape above the crate: which public exports are intended for Rust hosts, which helper types make calls feel Incan-like from Rust, and which support code owns repeated boundary mechanics. +RFC 031 created the first library artifact foundation: an Incan library can build a semantic manifest and implementation artifacts. The missing product-level answer is the shape above those artifacts: which public exports are intended for Rust hosts, which helper types make calls feel Incan-like from Rust, which support code owns repeated boundary mechanics, and which metadata defines compatibility without exposing generated Rust internals as API. -The missing piece is not only a command. It is a boundary. A Rust application embedding Incan code needs to know which calls are stable, how values convert, how errors surface, whether async calls need a runtime, whether panics are contained, how logs and telemetry are connected, and which compiler/runtime version produced the artifact. Without that boundary, users either treat generated Rust as hand-authored Rust or avoid Rust-hosted Incan entirely. +The missing piece is not only a command. It is a boundary. A Rust application embedding Incan code needs to know which calls are stable, how values convert, how errors surface, whether async calls need a runtime, whether panics are contained, how logs and telemetry are connected, and which compiler/runtime version produced the artifact. Without that boundary, users either treat compiler output as hand-authored Rust or avoid Rust-hosted Incan entirely. The end-state should be simple: an application team writes domain logic, policy, validation, transformations, routing decisions, or workflow steps in Incan, builds or publishes a Rust-facing package, and calls it from Rust as a typed dependency. The Rust app should remain in charge of process lifecycle, threading, deployment, and host resources. The Incan package should remain in charge of Incan language semantics and its exported behavior. ## Goals - Define a Rust-hosted caller model for native Rust applications that call Incan-authored libraries. -- Define a stable generated caller surface that builds on good generated Rust instead of hiding it. -- Make first-class generated Rust quality part of the Rust-hosted integration contract. +- Define a stable Rust-facing caller surface backed by ABI/package metadata. +- Keep implementation artifacts inspectable without making generated Rust source the public compatibility path. - Define how the `pub` system can express Rust-hosted public export profiles or facets. - Define conversion requirements for primitives, collections, models, enums, newtypes, results, options, and Rust-backed values. - Define reusable caller helpers that can reduce bespoke emitter output for common boundary shapes. - Define initialization, version, diagnostics, panic, async, logging, telemetry, and host capability responsibilities at the caller boundary. -- Preserve RFC 031's generated Rust crate as the concrete Cargo artifact. +- Preserve Cargo-native Rust host ergonomics without requiring generated Rust source to be the concrete public artifact. - Leave room for both local path development and published package consumption. - Keep Rust integration Rust-shaped enough to feel natural in Rust applications without making Incan source adopt Rust's full API design model. ## Non-Goals -- This RFC does not accept low-quality generated Rust as an implementation detail. The generated crate should remain readable and debuggable even when Rust hosts use the higher-level caller API. -- This RFC does not require generated Rust to look handwritten in every line. It requires generated Rust to be high-quality, documented where appropriate, idiomatic at its public surfaces, and stable enough for tooling and debugging. -- This RFC does not make every generated Rust module a stable public API. +- This RFC does not make generated Rust source the public package compatibility path. +- This RFC does not require every implementation backend to emit Rust source. +- This RFC does not make every generated Rust module a stable public API where generated Rust is still emitted. - This RFC does not replace `rust::` imports or Rust interop from Incan source. - This RFC does not define a C ABI, dynamic plugin ABI, `extern "C"` boundary, or cross-language FFI story. - This RFC does not require a Rust application to run the Incan compiler at runtime. @@ -106,14 +108,14 @@ The library is built for Rust-hosted consumption: incan build --lib --caller rust ``` -That command emits a normal Rust crate artifact with a generated caller module and metadata. A Rust application can then depend on it through Cargo: +That command emits or materializes a Cargo-usable caller artifact with caller metadata. A Rust application can then depend on it through Cargo: ```toml [dependencies] pricing_rules = { path = "../pricing_rules/target/lib" } ``` -The Rust application calls the generated typed wrapper rather than internal generated implementation details: +The Rust application calls the typed caller wrapper rather than internal implementation details: ```rust use pricing_rules::caller::{Caller, OrderInput}; @@ -129,7 +131,7 @@ fn price() -> Result<(), Box> { } ``` -For async entrypoints, the generated caller surface should make runtime requirements explicit: +For async entrypoints, the caller surface should make runtime requirements explicit: ```rust use pricing_rules::caller::{AsyncCaller, OrderInput}; @@ -145,7 +147,7 @@ async fn price_async() -> Result<(), Box> { } ``` -If an Incan export is not in the Rust-hosted public profile, Rust code may still see generated Rust implementation symbols, but those symbols are not promised as the host-facing API. The distinction is about stability and ergonomics, not about hiding bad Rust. +If an Incan export is not in the Rust-hosted public profile, Rust code must not rely on whatever implementation symbols happen to exist. The distinction is about semantic authority: caller metadata and caller APIs are stable; compiler implementation artifacts are not. The author-facing model is: @@ -153,7 +155,7 @@ The author-facing model is: Incan library source -> checked public Incan API -> Rust-hosted public profile - -> generated Rust crate + caller metadata + -> Rust-facing ABI/package metadata + caller artifact -> native Rust application ``` @@ -173,7 +175,7 @@ The caller boundary must include: The caller boundary must not require Rust consumers to import arbitrary compiler-generated implementation modules as the host API. Internal generated modules may exist and should remain readable, but only the caller namespace is stable for Rust-hosted consumption. -The caller boundary should be generated as part of the same Cargo package that contains the generated library crate unless a package format or registry mode explicitly separates implementation and caller crates. A Rust consumer must be able to depend on the artifact using ordinary Cargo dependency mechanics. +The caller boundary should be generated or materialized as a Cargo-usable artifact. It may live in the same package as implementation artifacts or in a sibling package, but Rust consumers must not need to know the compiler's internal implementation layout. Caller-visible Incan functions must have a representable Rust signature. The compiler must reject a Rust-hosted public export when any parameter, return value, type parameter, effect, or captured dependency cannot be represented by the caller boundary. @@ -201,19 +203,20 @@ Host capabilities used by caller-visible Incan code must be visible through meta ### Caller artifact shape -The caller artifact should be a Cargo-usable package. The simplest local layout is still the generated library crate from RFC 031, extended with a stable `caller` namespace and caller metadata. +The caller artifact should be a Cargo-usable package backed by Incan-owned caller metadata and ABI metadata. A current implementation may materialize that as a generated Rust package, but the normative contract is the Cargo-usable caller artifact and its metadata, not the emitted source layout. Conceptually, the package contains: ```text -generated Rust implementation stable caller namespace caller metadata +ABI/package metadata semantic manifest Cargo metadata +implementation artifact(s) ``` -The exact directory layout is not normative. The normative requirement is that Rust consumers do not need to know which files came from Incan source lowering and which files are support glue. +The exact directory layout is not normative. The normative requirement is that Rust consumers do not need to know which files came from Incan source lowering, backend emission, support glue, or ABI materialization. ### Support crate @@ -241,9 +244,9 @@ Caller type projection should prefer ordinary Rust types where doing so preserve | `Result[T, E]` | `Result` for domain result values | | `List[T]` | `Vec` | | `Dict[K, V]` | map type with documented ordering/hash requirements | -| `model` | generated Rust struct | -| `enum` | generated Rust enum | -| `newtype` | generated Rust newtype with checked construction | +| `model` | Rust caller struct | +| `enum` | Rust caller enum | +| `newtype` | Rust caller newtype with checked construction | Borrowed Rust signatures may be generated as an optimization, but the semantic contract must first be expressible with owned values. Borrowed projections must not expose Incan lifetime or ownership details as user-authored Incan concepts. @@ -258,23 +261,23 @@ For a function whose Incan signature returns `Result[Quote, PricingError]`, the ### Async and runtime policy -Async caller exports must not assume that the generated package owns the process runtime. The Rust host should either provide an async context by calling async functions or explicitly opt into a blocking wrapper that documents runtime behavior. +Async caller exports must not assume that the caller package owns the process runtime. The Rust host should either provide an async context by calling async functions or explicitly opt into a blocking wrapper that documents runtime behavior. Caller metadata should state whether an export is synchronous, async, blocking, or requires host-provided runtime services. This should compose with RFC 092 target and host capability metadata when those contracts mature. ### Diagnostics and observability -Caller failures should identify the caller export name, the Incan function name, and source-span metadata when available. Logging and telemetry should route through host-provided hooks where configured, rather than unconditionally initializing global logging from the generated package. +Caller failures should identify the caller export name, the Incan function name, and source-span metadata when available. Logging and telemetry should route through host-provided hooks where configured, rather than unconditionally initializing global logging from the caller package. ### Compatibility and migration -This RFC is additive. Existing `incan build --lib` consumers may continue depending directly on generated crates, but that should be documented as a lower-level artifact consumption path rather than the recommended Rust-hosted integration path. +This RFC is additive but reframes older generated-crate consumption as transitional. Existing `incan build --lib` consumers may continue depending directly on generated crates while that path exists, but that should be documented as a lower-level implementation-artifact path rather than the recommended Rust-hosted integration path. -Once caller artifacts exist, docs should steer Rust application authors toward caller APIs and reserve raw generated crate internals for debugging, compiler tests, or advanced toolchain integration. +Once caller artifacts exist, docs should steer Rust application authors toward caller APIs and reserve backend artifacts for debugging, compiler tests, inspection, or advanced toolchain integration. ## Alternatives considered -- **Tell Rust users to depend on the generated crate directly** — Rejected as the sole answer because generated Rust can be good Rust and still lack the right host-facing API profile, repeated boundary helpers, and stability story. +- **Tell Rust users to depend on the generated crate directly** — Rejected because it makes generated Rust internals the compatibility path. Rust hosts need a stable caller ABI/package contract even if the current backend happens to emit Rust. - **Use a dynamic plugin or C ABI boundary** — Rejected for this RFC because Incan already emits Rust, and Rust-hosted applications should get normal Cargo type checking, optimization, and dependency resolution. - **Use only a `build.rs` helper in the Rust application** — Useful for local development, but insufficient as the whole model because published artifacts and registry workflows should not require every consumer to run the Incan compiler. - **Make every public Incan export Rust-callable automatically** — Rejected as the default because Incan's `pub` system should be enriched with host-facing profiles instead of flattening every public Incan symbol into the same Rust-hosted contract. @@ -290,28 +293,28 @@ Once caller artifacts exist, docs should steer Rust application authors toward c ## Implementation architecture -The recommended architecture is to extend library builds with a caller adapter generation pass that consumes checked public API metadata and caller export declarations. The adapter should call into the generated implementation crate through stable internal paths chosen by the compiler, while exposing only the caller namespace to host Rust code. +The recommended architecture is to extend library builds with a caller adapter generation pass that consumes checked public API metadata, semantic facts, ABI metadata, and caller export declarations. The adapter should call into backend-owned implementation artifacts through compiler-owned internal paths or ABI entrypoints, while exposing only the caller namespace to host Rust code. -The support crate should remain narrow and versioned. Generated artifacts should declare the caller ABI version they were emitted against and validate it at initialization. Metadata should be inspectable by docs, LSP, and registry tooling so Rust-hosted integration can be documented and discovered without building the package. +The support crate should remain narrow and versioned. Caller artifacts should declare the caller ABI version they were emitted against and validate it at initialization. Metadata should be inspectable by docs, LSP, and registry tooling so Rust-hosted integration can be documented and discovered without building the package. Local development may later add a build-script helper that invokes the Incan compiler from a Rust workspace, but that helper should produce the same caller boundary as a prebuilt or published package. -Current package-facing characterization shows that ordinary `incan build --lib` artifacts can already expose owned scalar callable parameters through generated package exports, but borrowed non-`Copy` callable parameters are not yet consumable across a `pub::` package boundary. A producer export such as `Callable[Payload, None]` currently emits a Rust signature shaped like `fn(&Payload) -> ()`, while a downstream Incan consumer observer still emits `fn(Payload)`, causing Cargo type checking to fail. The caller adapter work must either generate a compatible borrowed wrapper for that boundary or reject/document the unsupported export before producing a broken consumer build. +Current package-facing characterization shows why generated implementation artifacts are not enough as the public contract. Ordinary `incan build --lib` artifacts can already expose owned scalar callable parameters through package exports, but borrowed non-`Copy` callable parameters are not yet consumable across a `pub::` package boundary. A producer export such as `Callable[Payload, None]` currently emits a Rust signature shaped like `fn(&Payload) -> ()`, while a downstream Incan consumer observer still emits `fn(Payload)`, causing Cargo type checking to fail. The caller adapter work must either generate a compatible borrowed wrapper for that boundary or reject/document the unsupported export before producing a broken consumer build. ## Layers affected -- **Library artifact model**: library builds must be able to include caller metadata and generated caller adapters alongside existing semantic manifests and generated Rust crates. +- **Library artifact model**: library builds must be able to include caller metadata, ABI/package metadata, caller adapters, and semantic manifests alongside backend implementation artifacts. - **Typechecker / API metadata**: caller export validation must prove that selected entrypoints and boundary types are representable for Rust-hosted calls. -- **IR Lowering / Emission**: generated Rust output must preserve a stable caller namespace and avoid making internal generated modules part of the Rust-hosted contract. +- **IR Lowering / Emission**: backend output must preserve a stable caller namespace or ABI entrypoint and avoid making internal generated modules part of the Rust-hosted contract. - **Stdlib / Runtime (`incan_stdlib`)**: host-facing runtime hooks, errors, logging, telemetry, async, and capability surfaces may need caller-compatible contracts. - **CLI / Tooling**: build commands should expose a caller artifact mode and diagnostics for unsupported caller exports. -- **LSP / Docs tooling**: tooling should surface caller-visible exports, generated Rust signatures, compatibility metadata, and unsupported-boundary diagnostics. +- **LSP / Docs tooling**: tooling should surface caller-visible exports, Rust-facing signatures, compatibility metadata, and unsupported-boundary diagnostics. - **Registry / Package metadata**: published packages should advertise whether they provide a Rust-hosted caller surface and which caller ABI version they require. ## Unresolved questions - What is the exact source syntax for marking caller-visible exports? -- Should caller adapters live in the same generated package as the implementation crate or in a sibling generated crate? +- Should caller adapters live in the same Cargo package as the implementation artifact or in a sibling package? - What is the first stable shape of the Rust support crate API? - Should synchronous wrappers around async Incan exports be generated by default, opt-in only, or disallowed? - How should nested domain results and boundary errors be represented ergonomically in Rust signatures? diff --git a/workspaces/docs-site/docs/RFCs/100_std_re_pythonic_regex.md b/workspaces/docs-site/docs/RFCs/100_std_re_pythonic_regex.md index 8a32181e8..794c5ba92 100644 --- a/workspaces/docs-site/docs/RFCs/100_std_re_pythonic_regex.md +++ b/workspaces/docs-site/docs/RFCs/100_std_re_pythonic_regex.md @@ -8,7 +8,7 @@ - RFC 023 (compilable stdlib and Rust module binding) - RFC 059 (`std.regex`) - RFC 070 (Result combinators) -- **Issue:** — +- **Issue:** https://github.com/dannys-code-corner/incan/issues/668 - **RFC PR:** — - **Written against:** v0.3 - **Shipped in:** — diff --git a/workspaces/docs-site/docs/RFCs/102_incan_semantic_layer_inspection_surface.md b/workspaces/docs-site/docs/RFCs/102_incan_semantic_layer_inspection_surface.md new file mode 100644 index 000000000..73c6c40e0 --- /dev/null +++ b/workspaces/docs-site/docs/RFCs/102_incan_semantic_layer_inspection_surface.md @@ -0,0 +1,376 @@ +# RFC 102: Incan Semantic Layer Inspection Surface + +- **Status:** Draft +- **Created:** 2026-05-23 +- **Author(s):** Danny Meijer (@dannymeijer) +- **Related:** + - RFC 015 (project lifecycle CLI) + - RFC 048 (checked contract metadata, Incan emit, and interrogation tooling) + - RFC 074 (template rendering and boilerplate provenance) + - RFC 075 (starter profiles and capability packs) + - RFC 076 (project mutation policy and recovery) + - RFC 077 (workspace and multi-package projects) + - RFC 078 (tool execution and typed workflow actions) + - RFC 079 (`incan.pub` artifact graph) + - RFC 080 (AI assets, models, prompts, evals, and agent metadata) + - RFC 082 (checked API documentation generation) + - RFC 085 (field metadata and type-shaped constraints) + - RFC 086 (schema descriptors and adapters) + - RFC 087 (reusable field contracts and model composition) + - RFC 092 (interactive runtime stdlib contracts) + - RFC 096 (declaration metadata blocks) + - RFC 097 (Rust-hosted Incan caller) +- **Issue:** https://github.com/dannys-code-corner/incan/issues/666 +- **RFC PR:** — +- **Written against:** v0.3 +- **Shipped in:** — + +## Summary + +This RFC defines the Incan Semantic Layer Inspection Surface: a local, versioned, machine-readable project model that joins checked source facts, project lifecycle facts, actions, capabilities, policy outcomes, provenance, artifacts, schema descriptors, AI assets, evals, and agent guidance into one inspectable contract for CLI, LSP, CI, docs tooling, registries, and agents. The goal is not to replace the subsystem RFCs that own those facts; the goal is to make their outputs converge into one semantic layer so tools do not scrape source files, generated Rust, manifests, README conventions, or unrelated command output to understand an Incan project. + +## Core model + +Read this RFC as nine foundations: + +1. **The semantic layer is local first:** the source of truth for project inspection is the local project or workspace, not a remote registry. +2. **Checked source facts and lifecycle facts meet in one model:** compiler-owned facts from RFC 048 and lifecycle-owned facts from RFC 074 through RFC 080 must be joinable through stable identities. +3. **Inspection is a product surface:** `incan inspect` or an equivalent command is a stable interface, not debug output. +4. **LSP is the proving consumer:** editor features should consume the same semantic layer as the CLI, CI, docs tooling, and agents. +5. **Human output is a view:** terminal prose may summarize inspection results, but machine-readable output is the canonical integration contract. +6. **Degraded states are explicit:** incomplete, stale, unsupported, unresolved, blocked, or policy-redacted facts must be represented directly instead of disappearing or being silently guessed. +7. **Agents are not privileged:** agent-facing data is the same data available to IDEs and CI, and agents may propose work but must not approve their own mutations. +8. **Graph explanation is required:** users and tools should be able to ask why a fact, action, artifact, policy outcome, or provenance edge exists. +9. **Subsystem RFCs keep ownership:** this RFC defines the aggregation and inspection contract, not the detailed semantics of templates, capabilities, actions, policy, AI assets, schemas, or registries. + +## Motivation + +Incan already has many of the ingredients of an intent and semantic layer. RFC 048 defines checked API and model metadata. RFC 074 defines template provenance. RFC 075 defines starters, capabilities, mutation plans, file roles, and agent guidance. RFC 076 defines policy outcomes. RFC 077 defines workspace inspection. RFC 078 defines typed actions. RFC 079 defines registry artifact relationships. RFC 080 defines AI assets and eval metadata. RFC 085, RFC 086, RFC 087, and RFC 096 deepen the model and schema contract. Each of those RFCs is useful on its own, but a tool that wants to understand a real project should not have to compose them through ad hoc command calls and local interpretation. + +The strategic risk is fragmentation. Incan can land every subsystem RFC and still fail to expose a coherent semantic layer if the facts remain scattered across separate commands, separate JSON shapes, separate sidecar files, and editor-specific glue. That would weaken the strongest product claim: Incan should be a language and toolchain where humans, compilers, IDEs, CI, documentation generators, registries, and agents can reason from the same project model. + +The practical problem appears first in the editor. A useful LSP should be able to show a checked declaration, the schema descriptor behind a model, the capability that created a file, the action that validates it, the policy that blocks a mutation, the generated artifact that depends on it, and the agent guidance that applies. If each of those answers comes from a different subsystem with different identity rules, editor tooling becomes a pile of partial integrations. The same is true for CI checks, documentation tooling, package browsers, and agent workflows. + +This RFC therefore makes the integration surface explicit. Incan should provide a local semantic inspection model that lets tools ask: what exists, what does it mean, what can run, what can mutate, what verifies it, what generated it, what depends on it, what policy applies, and what should an agent know before touching it? + +## Goals + +- Define a canonical local semantic inspection surface for Incan projects and workspaces. +- Define a versioned machine-readable semantic package format that can join compiler facts, project facts, lifecycle facts, and artifact facts. +- Define required stable identity classes for declarations, fields, modules, files, actions, capabilities, policies, generated artifacts, AI assets, evals, and graph edges. +- Define high-level command surfaces such as `incan inspect`, `incan graph explain`, and machine-readable LSP-facing equivalents without requiring exact final flag spelling. +- Define the relationship between RFC 048 checked metadata, RFC 074 template provenance, RFC 075 capabilities, RFC 076 policy, RFC 077 workspaces, RFC 078 actions, RFC 079 artifact graph data, and RFC 080 AI assets. +- Define how degraded, incomplete, unsupported, stale, blocked, and redacted facts are represented. +- Require CLI, LSP, CI, docs tooling, registry tooling, and agents to consume the same semantic facts where their needs overlap. +- Make agent-facing inspection an explicit stable integration target while preserving receiver-owned policy and approval boundaries. + +## Non-Goals + +- This RFC does not define a new source syntax. +- This RFC does not replace RFC 048 checked metadata, RFC 074 templates, RFC 075 capabilities, RFC 076 policy, RFC 077 workspaces, RFC 078 actions, RFC 079 registry graph semantics, or RFC 080 AI asset semantics. +- This RFC does not require a public `incan.pub` registry to exist before local inspection works. +- This RFC does not require every current or future artifact kind to be implemented before the first inspection surface ships. +- This RFC does not define the full LSP protocol mapping for every editor feature. +- This RFC does not allow agents to bypass policy, approval, sandboxing, or user review. +- This RFC does not require inspection commands to execute project code, run tools, fetch remote schemas, download models, or contact external services. +- This RFC does not make generated artifacts authoritative over checked source or checked metadata. +- This RFC does not standardize an on-disk semantic database format for compiler internals. + +## Guide-level explanation + +Users should be able to inspect an Incan project as a semantic object, not only as a folder of source files and manifests. + +```text +incan inspect --format json +``` + +The human-readable view might summarize the same model: + +```text +Project: checkout-console +Members: 3 +Capabilities: cli, testing.basic, schema.adapters +Actions: run, test, validate-schema, docs +Policy: source changes require review; remote AI execution blocked +Generated files: 4 tracked, 1 edited +AI assets: 1 prompt template, 2 eval suites +Warnings: schema adapter output is stale for model OrderSummary +``` + +The JSON output is the integration contract. A CI check, editor plugin, docs generator, or agent can consume the same data without scraping the terminal text. + +An editor can use the same model to power richer project affordances. Hovering a model field may show its checked type, field metadata, reusable field contract provenance, schema overlay facts, generated-doc status, and downstream adapter projections. Selecting a generated file may show which template or capability created it, whether it is bootstrap-owned or managed, and which update policy applies. Opening the command palette may show typed actions with risk and policy labels instead of generic shell scripts. + +Users and tools should also be able to ask why a relationship exists: + +```text +incan graph explain model:OrderSummary.status +incan graph explain action:validate-schema +incan graph explain artifact:target/schema/order_summary.json +``` + +Example human-readable explanation: + +```text +model:OrderSummary.status + declared by source model OrderSummary + imports reusable field contract order_status + appears in schema overlay WarehouseOrder + validates generated artifact target/schema/order_summary.json + affected actions: validate-schema, docs + policy: source metadata changes require review +``` + +The same explanation should be available as structured data so LSP, CI, docs tooling, and agents can present it in their own UI. + +For agents, the model is a bounded context source. An agent can discover relevant files, capabilities, actions, tests, evals, policy restrictions, and generated artifact provenance before proposing a patch. The agent still cannot approve its own mutation, execute hidden lifecycle hooks, or infer permissions from guidance text. + +## Reference-level explanation + +### Semantic package + +The semantic inspection surface must expose a versioned semantic package. The exact JSON field names are not normative in this Draft, but the package must identify: + +- semantic package schema version; +- Incan toolchain version; +- project or workspace root identity; +- selected workspace scope when applicable; +- source snapshot identity when available; +- project manifest facts; +- lockfile and dependency facts when available; +- checked source declarations from RFC 048; +- contract-backed model facts from RFC 048; +- field metadata, reusable field provenance, and schema descriptor facts from RFC 085, RFC 086, RFC 087, and RFC 096 where available; +- file roles, capability status, capability provenance, template provenance, and generated-file ownership from RFC 074 and RFC 075; +- typed actions from RFC 078; +- policy outcomes from RFC 076; +- workspace topology from RFC 077; +- artifact graph and registry relationship facts from RFC 079 when available locally; +- AI asset, prompt, eval, and agent guidance facts from RFC 080 when available; +- diagnostics, warnings, degraded states, and redactions. + +The semantic package must not require remote registry access for basic local inspection. Remote or registry-backed facts may appear when they are already available in project state, package artifacts, lockfiles, cached descriptors, or explicitly requested registry queries. + +### Command surface + +The CLI must provide a project inspection command. The recommended spelling is: + +```text +incan inspect --format json +``` + +The exact final spelling may change, but the command must expose the semantic package in a documented machine-readable format. + +The CLI should provide a graph explanation command. The recommended spelling is: + +```text +incan graph explain --format json +``` + +Selectors should support at least declarations, model fields, files, actions, capabilities, generated artifacts, policy decisions, and AI assets when those objects are present in the semantic package. + +Existing subsystem commands such as action listing, capability status, policy checks, workspace inspection, metadata extraction, and template status may continue to exist. Their machine-readable output should either embed compatible semantic package fragments or reference the same stable identities used by the semantic package. + +### Stable identities + +The semantic package must represent stable identities for objects that other tools need to join. This RFC requires stable identities for at least: + +- project and workspace members; +- modules and public declarations; +- model fields and reusable field contracts; +- schema descriptors and overlays; +- source files and generated files; +- templates and template provenance records; +- capabilities and applied capability records; +- actions and action providers; +- policy decisions and risk categories; +- package artifacts and generated artifacts; +- AI assets, prompt templates, evals, datasets, and agent guidance records. + +Stable identities must be deterministic for a given source and project state. They must not depend on process memory addresses, nondeterministic traversal order, or human-formatted output. + +When an identity cannot be made stable, the semantic package must mark it as unstable or local-only. Tools must not treat unstable identities as durable cross-run anchors. + +### Edges + +The semantic package must represent relationships as first-class edges where possible. This RFC requires support for these relationship kinds: + +- `declares`: source or artifact declares a semantic object; +- `materializes`: contract metadata materializes a model or declaration; +- `generates`: template, capability, action, or adapter generates a file or artifact; +- `validates`: action, test, eval, or policy validates an object; +- `depends-on`: object depends on another object; +- `provided-by`: package, capability, or artifact provides an object; +- `applies-policy`: policy decision applies to an action, mutation, artifact, or source; +- `created-by-capability`: file, action, or metadata originated from a capability; +- `projects-from`: generated schema, docs, or adapter output projects from checked descriptors; +- `guided-by`: agent guidance applies to a file role, capability, action, or project shape. + +Implementations may add extension edge kinds. Unknown edge kinds must remain visible in machine-readable output and must not be silently dropped by generic consumers. + +### Degraded and partial facts + +The semantic package must represent degraded states explicitly. Useful states include: + +- `complete`: the fact is fully checked and current; +- `partial`: the fact is present but incomplete; +- `unsupported`: the toolchain knows the object exists but cannot inspect it fully; +- `stale`: the fact was derived from an older source state; +- `blocked`: policy or configuration prevents resolving the fact; +- `redacted`: the fact exists but sensitive content is intentionally hidden; +- `unknown`: the toolchain cannot determine whether the fact exists. + +For degraded facts, the package should include a reason code and a human-readable diagnostic where possible. Consumers must not infer absence from a missing optional field when a degraded state is available. + +### Policy and approval + +Policy outcomes from RFC 076 must be represented in the semantic package when policy is evaluated. Inspection may report policy status without applying mutations or running actions. + +Agent guidance, AI assets, action descriptors, template provenance, and capability metadata must not grant approval. The semantic package may help an agent propose a patch or select a workflow, but approval remains governed by RFC 076 and the receiving project. + +Sensitive values must follow the redaction rules of the owning subsystem. For example, template parameters marked sensitive must not appear as raw values in inspection output, and remote AI configuration must not expose secrets. + +### LSP consumption + +The LSP should treat the semantic package as the editor-facing project model where practical. It may cache or request focused views, but it should not reimplement independent logic for capability status, action discovery, policy outcomes, generated-file provenance, schema descriptors, or agent guidance. + +Editor features that should consume this surface include: + +- project tree grouping by file role and generated-file ownership; +- hover and go-to-definition for checked declarations, aliases, partials, fields, reusable field contracts, schema overlays, and generated artifacts; +- action buttons for typed actions with risk and policy labels; +- diagnostics for stale generated files, blocked policy, unsupported actions, invalid capability state, and stale schema projections; +- code actions for reviewable capability, template, or generated artifact updates; +- agent guidance discovery without executing agents or hidden prompts. + +The LSP may expose focused protocol-specific requests rather than returning the full semantic package on every editor operation. Those focused responses must preserve the same identities and degraded-state semantics as the CLI inspection surface. + +### CI, docs, registry, and agent consumption + +CI tools should be able to consume the semantic package to select typed actions, enforce policy checks, verify generated artifact freshness, run relevant evals, and fail on stale or unsupported project states. + +Documentation tooling should be able to consume checked declarations, schema descriptors, contract metadata, capability docs links, generated-file provenance, and artifact relationships from the semantic package instead of parsing source or generated Rust. + +Registry and package tooling may consume exported semantic package fragments when publishing packages or building artifact cards, but remote registries must not become the local authority for project mutation. + +Agentic tooling may consume the semantic package to identify relevant files, tests, evals, actions, capabilities, and constraints. It must treat policy outcomes, risk categories, and degraded states as binding context for proposal generation. + +## Design details + +### Relationship to RFC 048 + +RFC 048 remains the owner of checked API metadata and contract-backed model metadata. This RFC treats RFC 048 facts as compiler-owned source facts inside the larger semantic package. + +The semantic package must not weaken RFC 048 by falling back to source-text scraping or generated Rust inspection when checked metadata is available. If checked metadata cannot be produced because the source has parse or type errors, the semantic package must report degraded source facts and diagnostics. + +### Relationship to RFC 074 and RFC 075 + +RFC 074 owns template rendering and provenance. RFC 075 owns starter and capability descriptors, application, mutation planning, file roles, tooling metadata, and agent guidance metadata. This RFC joins their records into the local semantic graph. + +Capability and template state must remain explicit project tooling state. The semantic package must not infer that a file is generated merely because it resembles a known template. + +### Relationship to RFC 076 + +RFC 076 owns policy evaluation and approval semantics. This RFC requires policy results to be surfaced through the semantic package, but does not define policy rules. + +When policy has not been evaluated for an object, the semantic package must distinguish `not-evaluated` from `allow`. Lack of a policy result must not be treated as permission. + +### Relationship to RFC 077 + +RFC 077 owns workspace topology and scoped mutation planning. This RFC requires semantic inspection to include selected workspace scope and member identity so tools do not accidentally treat whole-workspace facts as single-member facts. + +### Relationship to RFC 078 + +RFC 078 owns typed action semantics, source resolution, execution modes, risk labels, dry-run behavior, and invocation. This RFC requires actions to appear as semantic objects with stable identities and graph edges to inputs, outputs, providers, policy outcomes, evals, and generated artifacts where available. + +### Relationship to RFC 079 + +RFC 079 owns the registry artifact graph. This RFC owns the local project semantic graph. The two graphs should share compatible artifact kinds, relationship vocabulary, and identity references where practical, but the local semantic graph must work without a public registry. + +Registry metadata may enrich local inspection, but it must not replace receiver-owned planning, policy, or mutation authority. + +### Relationship to RFC 080 + +RFC 080 owns AI asset metadata, prompt templates, datasets, evals, agent guidance, and local/cloud execution constraints. This RFC requires those facts to appear in inspection output when they are project-relevant and available. + +Prompt templates and system messages that affect project behavior must be inspectable as artifacts. Agent guidance must remain descriptive and must not cause implicit agent execution. + +### Relationship to RFC 085, RFC 086, RFC 087, and RFC 096 + +Those RFCs own field metadata, schema descriptors, reusable field contracts, model composition, and declaration metadata blocks. This RFC requires their normalized checked facts and provenance edges to be visible through the semantic package where supported. + +Adapter outputs must remain projections of checked descriptors, not source truth. The semantic package should preserve edges from adapter outputs back to descriptor identities when available. + +### Relationship to RFC 092 and RFC 097 + +RFC 092 owns interactive runtime target manifests and host capability contracts. RFC 097 owns the Rust-hosted caller boundary. This RFC allows those emitted manifests, host capability facts, Rust-facing ABI/caller artifacts, and caller metadata to appear in the semantic package when available, especially for LSP, docs, CI, and registry inspection. + +## Alternatives considered + +### Keep subsystem JSON outputs independent + +Rejected because it preserves fragmentation. Independent outputs can be useful, but they must share identities and be joinable through a canonical project model. + +### Make the LSP the only integration owner + +Rejected because CI, docs tooling, registry tooling, and agents need the same facts outside an editor. LSP is the proving consumer, not the source of truth. + +### Put the semantic layer in `incan.pub` + +Rejected because local projects must remain inspectable without registry access, and local tooling owns receiver-side mutation plans and policy. Registry graph metadata can enrich inspection but must not be required for it. + +### Use generated Rust as the inspection source + +Rejected because Incan semantics include source-level facts, metadata, provenance, policy, capabilities, and actions that generated Rust either cannot represent or should not be authoritative for. + +### Treat agent guidance as separate from normal tooling + +Rejected because giving agents a special path would create drift and privilege confusion. Agents should consume the same semantic facts as IDEs and CI, subject to the same policy boundaries. + +## Drawbacks + +This RFC adds an integration obligation across many subsystems. Each subsystem must preserve identities and enough structured data for the semantic package, which can slow early implementation. + +A broad semantic package can become too large or too slow if every command eagerly computes every fact. Implementations will need focused views, lazy computation, or scope selection while preserving the same identity and degraded-state contract. + +Versioning the inspection schema creates compatibility work. Once tools and agents depend on the JSON shape, changes need migration discipline. + +There is a risk of overpromising if implementation work tries to expose every artifact kind at once. Implementation sequencing should prove the local compiler and lifecycle join while preserving the full 1.0 contract described by this RFC. + +## Implementation architecture + +This section is non-normative. + +A practical implementation shape is to treat the semantic inspection surface as a join over two fact domains: + +- compiler facts: modules, declarations, types, contracts, diagnostics, checked metadata, schema descriptors, and stable source identities; +- project facts: manifests, workspaces, lock state, capabilities, actions, templates, generated-file provenance, policy, artifacts, AI assets, and registry-derived local metadata. + +The join should happen through stable identities and graph edges rather than by embedding subsystem-specific blobs that consumers must reinterpret. Subsystems may still own their specialized payloads, but the semantic package should expose enough shared fields for generic tooling to navigate the project. + +Implementations should support focused queries so LSP and CI can request only the facts they need. Focused query output should remain a semantic package fragment with the same schema version, identity rules, degraded-state model, and edge vocabulary as full inspection output. + +## Layers affected + +- **Compiler semantic analysis**: must expose checked source facts, diagnostics, stable identities, and degraded states in a form that the semantic package can consume. +- **Project model / lifecycle tooling**: must expose manifest, workspace, lock, capability, action, template, policy, provenance, and AI asset facts through shared identities. +- **CLI / tooling**: must provide machine-readable inspection and graph explanation commands, plus focused views where needed. +- **LSP / IDE tooling**: should consume semantic package facts for project views, hovers, definitions, diagnostics, run actions, generated-file status, policy status, and agent guidance discovery. +- **Docs tooling**: should consume checked declarations, schema descriptors, provenance, and artifact edges from the semantic package where useful. +- **CI / automation**: should consume action, policy, stale-artifact, eval, and degraded-state facts without parsing human output. +- **Registry / package integration**: should map local artifact identities and relationship edges to registry artifact graph metadata when publishing or inspecting packages. +- **Agentic tooling**: may consume the semantic package for context selection and proposal generation, but must respect policy outcomes and approval boundaries. + +## Unresolved questions + +- Should the canonical command be `incan inspect`, `incan project inspect`, `incan graph inspect`, or another spelling? +- Should graph explanation be a subcommand of inspection, such as `incan inspect explain`, or a separate `incan graph explain` command? +- Which semantic package schema fields are mandatory for the 1.0 north-star contract, and which unsupported domains should appear as explicit degraded facts until their owning RFCs land? +- Which identity formats should be stable across machines, packages, and versions, and which should be explicitly local-only? +- Should focused LSP queries use the same JSON schema directly or a protocol-specific projection that preserves semantic package identities? +- How should semantic package fragments be cached and invalidated without standardizing compiler-internal storage? +- Should exported package artifacts embed a semantic package fragment, or should they embed only RFC 048 metadata plus artifact graph metadata until a later publishing RFC? +- What compatibility policy should apply when an older tool consumes a newer semantic package with unknown object or edge kinds? + + diff --git a/workspaces/docs-site/docs/RFCs/103_secret_values.md b/workspaces/docs-site/docs/RFCs/103_secret_values.md new file mode 100644 index 000000000..2c74b2ccf --- /dev/null +++ b/workspaces/docs-site/docs/RFCs/103_secret_values.md @@ -0,0 +1,319 @@ +# RFC 103: `std.secrets` — Secret strings, secret bytes, and redaction-safe values + +- **Status:** Draft +- **Created:** 2026-05-24 +- **Author(s):** Danny Meijer (@dannymeijer) +- **Related:** + - RFC 017 (validated newtypes with implicit coercion) + - RFC 033 (`ctx` typed configuration context) + - RFC 066 (`std.http` HTTP client surface) + - RFC 072 (`std.logging` structured logging) + - RFC 078 (tool execution and typed workflow actions) + - RFC 089 (`std.environ` runtime environment access) + - RFC 090 (typed CLI framework) + - RFC 093 (`std.telemetry` observability) + - RFC 102 (semantic layer inspection surface) +- **Issue:** https://github.com/dannys-code-corner/incan/issues/661 +- **RFC PR:** - +- **Written against:** v0.3 +- **Shipped in:** — + +## Summary + +This RFC proposes `std.secrets` as Incan's standard library home for secret value wrappers, beginning with `SecretStr` and `SecretBytes`. Secret values are ordinary typed values that can flow through config, CLI, environment, HTTP, logging, telemetry, workflow actions, and generated reports without revealing their plaintext through unauthorized display, debug, structured logs, diagnostics, default serialization, or inspection surfaces. The goal is not to pretend secrets become impossible to copy or exfiltrate inside a compromised process; the goal is to make plaintext exposure deny-by-default, keep raw access scoped and intentional, and allow stronger protected storage such as encrypted idle memory where the backend can provide it. + +## Core model + +1. **Secrets are values, not logging conventions:** secrecy must travel with the value's type so redaction is not rebuilt separately by every caller. +2. **Plaintext exposure is deny-by-default:** Incan-owned display, debug output, logs, telemetry attributes, diagnostics, semantic inspection, reports, and default serialization must not reveal secret contents. +3. **Reveal is scoped and intentional:** APIs that need raw bytes or strings should consume `SecretStr` or `SecretBytes` directly, or require an intentionally named scoped reveal operation that tooling can recognize. +4. **Protected idle storage is preferred:** implementations should keep secret contents encrypted or otherwise protected while idle when a backend can do so meaningfully, and decrypt only inside a scoped reveal operation. +5. **Memory guarantees are honest:** protected idle storage and zeroization reduce exposure, but the public contract must not promise that every intermediate copy made by encoders, transport backends, operating systems, foreign APIs, crash handlers, or the process itself is erased. +6. **Specific types come first:** `SecretStr` and `SecretBytes` are the initial stable surface. A generic `Secret[T]` may come later if it does not weaken the concrete-string and concrete-bytes contracts. +7. **Tooling preserves sensitivity metadata:** CLI, LSP, semantic inspection, workflow action output, generated docs, and reports should know that a value exists and what type it has without seeing the raw payload. + +## Motivation + +Python ecosystems often represent secrets with wrapper classes, Pydantic field flags, logging filters, and framework-specific conventions. Those mechanisms help, but they remain easy to bypass because Python string interpolation, `repr`, dictionaries, serializers, exception traces, and third-party clients can all treat the wrapped value as just another object unless every boundary cooperates perfectly. + +Incan has a better opportunity because its stdlib, typechecker, generated Rust, structured logging, HTTP surface, CLI framework, environment access, action metadata, and semantic inspection model can agree on one value-level contract. A `SecretStr` used as a CLI option, loaded from an environment variable, passed to an HTTP authorization helper, logged as a structured field, or surfaced in an action report should remain recognizably present but redacted all the way through those boundaries. The core promise should be stronger than "nice `repr`": plaintext must not leave a secret wrapper through an Incan-owned surface unless the code has made an explicit reveal decision or passed the value to a trusted API that owns a scoped reveal internally. + +This RFC also closes a design gap left deliberately open by RFC 017. Validated newtypes can model domain-specific string and byte constraints, but secret handling is more than a validation constraint: it changes display, debug, logging, diagnostic serialization, wire-boundary APIs, equality, cloning, and drop behavior expectations. + +## Goals + +- Add a `std.secrets` module with `SecretStr` and `SecretBytes`. +- Make redaction a property of the value type rather than a per-logger or per-HTTP-client convention. +- Prevent plaintext secret emission through Incan-owned display, debug, diagnostic, logging, telemetry, semantic inspection, generated-report, and default serialization paths. +- Require safe default behavior for display, debug, structured logs, telemetry, diagnostics, semantic inspection, and generated reports. +- Provide intentionally named, tooling-visible APIs for scoped exposure of raw secret material at trusted boundaries. +- Prefer encrypted or otherwise protected idle memory for secret storage where the target backend can provide it meaningfully. +- Let stdlib consumers such as `std.http`, `std.environ`, typed CLI surfaces, `ctx`, workflow actions, logging, and telemetry accept or preserve secret values without converting them to plain `str` or `bytes`. +- Define a conservative serialization contract that prevents accidental JSON, TOML, YAML, CLI, or report emission of raw secret contents. +- Define honest memory-handling expectations, including scoped plaintext lifetimes and best-effort zeroization for plaintext buffers where the backend can support it. +- Leave room for future secret providers, vault integrations, redaction policies, and generic secret wrappers without blocking the concrete `SecretStr` and `SecretBytes` surface. + +## Non-Goals + +- This RFC does not define a password manager, vault, keyring, or secrets backend. +- This RFC does not define encryption at rest for source files, manifests, lockfiles, logs, reports, or generated artifacts. +- This RFC does not provide full information-flow control, taint tracking, or a data-loss-prevention system. +- This RFC does not guarantee that all process memory, operating-system buffers, network buffers, allocator copies, panic payloads, crash dumps, foreign library copies, or compiler temporaries are erased. +- This RFC does not claim that encrypted idle storage protects against arbitrary code execution inside the same process; any implementation must still hold or derive decryption material somewhere. +- This RFC does not make secrets safe to expose to untrusted code. +- This RFC does not define random secret generation; a future `std.random` or expanded `std.secrets` surface may do that separately. +- This RFC does not define identity protocols such as SAML, OAuth, OIDC, JWT validation, service-account exchange, or single sign-on workflows. +- This RFC does not standardize every sensitive-data class such as PII, payment data, access tokens, API keys, passwords, and private keys as distinct semantic categories in the initial surface. +- This RFC does not replace access control, capability checks, sandboxing, policy approval, or runtime permission boundaries. + +## Guide-level explanation + +Users should be able to load a secret value and pass it through normal code without turning it into a plain string just to keep working. + +```incan +from std.environ import env +from std.secrets import SecretStr + +token: SecretStr = env.secret_str("SERVICE_TOKEN")? +println(token) +``` + +The printed value is redacted. The exact placeholder is a design detail, but it must not include the token. + +HTTP clients and other stdlib APIs should accept secret values directly: + +```incan +from std.environ import env +from std.http import Client, bearer +from std.secrets import SecretStr + +token: SecretStr = env.secret_str("SERVICE_TOKEN")? +client = Client(default_headers={"Authorization": bearer(token)}) +response = client.get("https://api.example.com/items")? +``` + +The caller does not reveal the token manually. The HTTP boundary may perform a scoped internal reveal when constructing the wire request, but diagnostics, debug output, retries, telemetry, and action reports must preserve sensitivity. + +When a raw value is genuinely needed, the operation should read as intentional and scoped: + +```incan +from std.secrets import SecretBytes + +def sign_with_key(raw_key: bytes) -> Signature: + return hmac.sign(raw_key, payload) + + +key: SecretBytes = SecretBytes.from_hex(env.secret_str("SIGNING_KEY_HEX")?)? +signature = key.with_exposed_bytes(sign_with_key) +``` + +The exact reveal method names remain open in this Draft. The important property is that code review, search, LSP, and policy tooling can recognize raw-secret exposure sites, and that the preferred shape does not hand an ordinary string or byte buffer back to the caller for uncontrolled storage. + +Secret values should also compose with typed configuration and CLIs: + +```incan +from std.secrets import SecretStr + +ctx Deploy: + api_token: SecretStr = env("API_TOKEN") + endpoint: str = "https://api.example.com" +``` + +An inspection view can show that `api_token` exists, is required, and has type `SecretStr`, without showing the token itself. + +## Reference-level explanation + +### Module surface + +`std.secrets` must expose `SecretStr` and `SecretBytes`. + +`SecretStr` must represent owned UTF-8 secret text. `SecretBytes` must represent owned binary secret material. + +The module may expose helper types such as redaction placeholders, reveal guards, redacted serialization adapters, or sensitivity metadata, but `SecretStr` and `SecretBytes` are the required initial surface. + +### Construction + +`SecretStr` must be constructible from a `str` through an explicit constructor or conversion path. `SecretBytes` must be constructible from `bytes` through an explicit constructor or conversion path. + +Construction APIs should make plain-to-secret conversion visible in source. Implicit conversion from `str` to `SecretStr` or from `bytes` to `SecretBytes` should be avoided unless a surrounding API already declares that an input position is secret, such as a typed CLI option, an environment accessor, or a `ctx` field. + +`SecretStr` should support conversion to `SecretBytes` using an explicit encoding operation. `SecretBytes` should support UTF-8 decoding into `SecretStr` through a fallible operation. + +`std.environ` should provide secret-returning helpers, such as a `secret_str` shape, so callers do not need to load an environment variable as plain text and then wrap it manually. + +### Display and debug behavior + +`SecretStr` and `SecretBytes` must redact their contents in display, debug, assertion failure, panic, diagnostic, and structured-inspection contexts owned by the Incan standard library and toolchain. + +The redacted representation must communicate that the value is secret and present. It must not include the secret contents, prefix, suffix, length, checksum, entropy estimate, or other derived value unless a later RFC defines an explicit policy for such metadata. + +String interpolation and formatting protocols must use the redacted representation by default. Formatting a secret must not implicitly call the reveal operation. + +### Plaintext leakage boundary + +The normative security boundary for this RFC is Incan-owned plaintext emission. `SecretStr` and `SecretBytes` must not reveal raw contents through Incan-owned display, debug, panic formatting, assertion messages, diagnostics, structured logs, telemetry attributes, semantic inspection, generated reports, CLI help, CLI echo, default serialization, or action metadata. + +This boundary also applies to nested structures. A model, list, dict, result, error, request, response, action input, or telemetry event containing a secret value must preserve redaction when formatted or serialized through Incan-owned mechanisms. + +Trusted stdlib APIs may reveal plaintext internally only for the duration of the operation that requires it, such as computing an HMAC or sending an HTTP authorization header. That internal reveal must not become observable through error values, debug payloads, telemetry attributes, retry reports, or generated artifacts. + +### Reveal operations + +`SecretStr` must provide an intentionally named operation for exposing the raw `str` value. `SecretBytes` must provide an intentionally named operation for exposing the raw bytes value. + +Reveal operations must be easy for tooling to identify. They should use names that communicate risk, such as `expose_secret`, `expose_secret_str`, or `expose_secret_bytes`, rather than neutral names like `value`, `get`, or `as_str`. + +The preferred reveal shape is scoped: a callback, guard, or equivalent API that makes plaintext available only for a bounded lexical or dynamic lifetime. Owned plaintext copies should either be unavailable by default or exposed through a more explicit and noisier escape hatch than scoped reveal. + +The reveal operation may return a borrowed view, a scoped guard, a backend-specific safe-access wrapper, or an owned copy only when the API name makes the copying behavior explicit. The accepted design must document the lifetime, copying behavior, and zeroization behavior of every reveal path. + +APIs that genuinely need raw material should prefer accepting `SecretStr` or `SecretBytes` directly instead of forcing user code to reveal the secret first. + +### Serialization + +Default data serialization of `SecretStr` and `SecretBytes` must not emit raw secret contents. + +For diagnostic serialization, generated reports, semantic inspection, logs, telemetry, and CLI output, the value must serialize as a redacted secret marker or an equivalent structured redaction object. + +For data formats that are intended to leave the process as user data, such as JSON request bodies, TOML files, YAML files, or generated artifacts, default serialization should fail unless the caller chooses an explicit redacted adapter or an explicit reveal operation. This avoids accidentally sending placeholder text where a real secret was expected, and avoids accidentally persisting the raw value. + +### Equality, ordering, and hashing + +`SecretStr` and `SecretBytes` should not expose ordering operations by default. + +Equality is an open design question. If equality is exposed, it should avoid timing behavior that is obviously inappropriate for token, password, or key comparison, and the docs must state whether the comparison is constant-time. If the implementation cannot provide a meaningful constant-time guarantee for a given storage representation, it should prefer an explicit comparison helper over ordinary equality. + +Hashing secret values should be avoided by default because hash maps and debug tooling often make key material harder to reason about. If hash support is needed later, it should be introduced deliberately with documented semantics. + +### Cloning and copying + +`SecretStr` and `SecretBytes` must not be trivially copyable value types. + +Cloning may be supported when the language's ownership model requires it for ordinary value flow, but clone operations must preserve secrecy metadata and must not reveal raw contents. The docs must state that cloning creates another copy of the secret material. + +### Protected storage and memory handling + +Implementations should keep secret contents encrypted or otherwise protected while idle when the target backend can provide a meaningful protected-storage implementation. Plaintext should be produced only inside scoped reveal operations or trusted stdlib internals that need raw bytes or text for a bounded operation. + +Any protected-storage implementation must document its threat model. Encrypting a buffer while idle can reduce accidental plaintext retention and may help with some memory disclosure scenarios, but it does not protect against arbitrary code execution in the same process, a compromised runtime, a debugger with full process access, or backend APIs that must receive plaintext. + +Plaintext buffers created during reveal should be zeroized as soon as their scoped use ends when the backend can support that. `SecretBytes` should zeroize owned plaintext memory on drop when generated code can do so without weakening correctness. `SecretStr` may also zeroize owned storage when implemented over a mutable owned buffer, but the public contract must not imply that all UTF-8 string copies are erased. + +Both types must document that redaction is an exposure-control guarantee for standard display, debug, logging, telemetry, diagnostics, and serialization paths. Protected idle storage and zeroization strengthen that guarantee, but they are not full memory-forensics or same-process-compromise guarantees. + +The implementation should avoid unnecessary copies in stdlib APIs that consume or forward secret values, especially HTTP authorization helpers, cryptographic helpers, and secret-provider integrations. + +### Logging, telemetry, diagnostics, and inspection + +`std.logging`, `std.telemetry`, diagnostics, and semantic inspection must treat `SecretStr` and `SecretBytes` as sensitive fields by type. + +Structured outputs should preserve the fact that a field exists, its declared type, and relevant non-sensitive metadata such as source kind when appropriate. They must not include the raw value. + +Tooling should mark explicit reveal operations as searchable and inspectable sites. LSP hover, semantic inspection, and policy checks may use those sites to explain where secret material leaves the protected wrapper. + +### HTTP and wire-boundary APIs + +`std.http` authorization helpers, header builders, request diagnostics, retry reporting, and telemetry should preserve secret sensitivity. Header values constructed from `SecretStr` or `SecretBytes` must be redacted in debug-facing output even if the header name is not in a built-in sensitive-header list. + +`std.http` may expose raw secret material internally when sending a request. That internal exposure must not change the public `Request`, `Response`, `HttpError`, log, telemetry, or action-output redaction contract. + +### Typed actions, CLIs, and configuration + +Typed action inputs, CLI options, and `ctx` fields should be able to declare `SecretStr` and `SecretBytes` directly. + +Machine-readable action metadata should distinguish a required secret input from a plain string input. Action output must not include raw secret values unless a future policy system defines an explicit, user-approved reveal path. + +CLI help may show that an option expects a secret. It must not echo secret defaults or environment-derived values. + +### Higher-level identity protocols + +Identity and federation protocols such as SAML, OAuth, OIDC, JWT validation, service-account exchange, and single sign-on workflows should be built above `std.secrets`, not inside it. Those protocols have their own security models: XML or JSON token formats, signatures, certificates, issuer and audience validation, replay windows, metadata discovery, clock skew, session state, and provider-specific policy. + +`std.secrets` should provide the primitive secret value contract those packages consume. A future identity or platform library may store private keys, bearer tokens, client secrets, SAML assertions, or signed credentials in `SecretStr` or `SecretBytes`, and may use scoped reveal internally when validating or transmitting them. That does not make `std.secrets` responsible for the protocol semantics. + +## Design details + +### Syntax + +This RFC does not introduce new parser syntax. `SecretStr` and `SecretBytes` are stdlib types. + +### Semantics + +Secret values have ordinary type identity and can be passed, returned, stored in models, and used in containers according to the language's normal value rules. Their special behavior is attached to display, debug, formatting, serialization, logging, telemetry, diagnostics, inspection, equality, hashing, cloning, reveal, protected storage, and drop semantics. + +Implicit downcast from `SecretStr` to `str` and from `SecretBytes` to `bytes` must not be allowed. Raw exposure must require either an explicit scoped reveal operation or a trusted stdlib API that accepts a secret type directly and owns the scoped reveal internally. + +### Interaction with existing features + +- **RFC 017 (validated newtypes)**: secret values may use newtype-like machinery internally, but their display, debug, serialization, and memory expectations are a separate contract. +- **RFC 033 (`ctx`)**: typed configuration can declare secret fields and source them from environment or future secret providers without exposing raw values in inspection. +- **RFC 066 (`std.http`)**: HTTP auth helpers and headers should accept secret values and preserve redaction through request diagnostics, retries, telemetry, and workflow output. +- **RFC 072 (`std.logging`)**: structured logging should redact secret-typed fields by default. +- **RFC 078 (typed workflow actions)**: action inputs and outputs should preserve sensitivity metadata so reports can describe secret use without exposing values. +- **RFC 089 (`std.environ`)**: environment access should provide secret-returning helpers that avoid plain-string staging. +- **RFC 090 (typed CLI framework)**: CLI options can use `SecretStr` and `SecretBytes` as declared types. +- **RFC 093 (`std.telemetry`)**: telemetry attributes and events must redact secret-typed values. +- **RFC 102 (semantic layer inspection surface)**: semantic inspection should represent secret facts as redacted facts with stable type and source metadata. + +### Compatibility / migration + +This feature is additive. Existing code that stores tokens in plain strings remains valid, but docs and examples should prefer `SecretStr` and `SecretBytes` at configuration, CLI, environment, HTTP, and action boundaries once the types exist. + +Migration helpers may wrap existing `str` or `bytes` values explicitly. Such helpers should not hide the fact that code still created a plain value before wrapping it. + +## Alternatives considered + +- **Plain `newtype str` and `newtype bytes` only** + - Rejected because newtypes alone do not define formatting, debug, serialization, logging, telemetry, equality, cloning, and memory behavior. +- **Logging-only redaction** + - Rejected because secrets leak through more than logs: debug strings, exception messages, assertions, generated reports, telemetry, HTTP diagnostics, CLI echo, and semantic inspection all matter. +- **HTTP-only secret headers** + - Rejected because the same token often starts in environment or CLI config, flows through `ctx`, enters an HTTP client, appears in telemetry, and may be referenced by typed actions. +- **One generic `Secret[T]` as the first surface** + - Rejected for the initial version because strings and bytes have distinct encoding, display, comparison, and memory concerns. A generic wrapper may still be useful later. +- **Always serialize redacted placeholders** + - Rejected for data serialization because silently writing `` into JSON payloads, config files, or generated artifacts can create corrupt data and hide bugs. +- **Unscoped raw getters** + - Rejected because a method that returns an ordinary `str` or `bytes` as the primary reveal path makes it too easy to store, log, serialize, or return plaintext accidentally. +- **Always require manual reveal before wire use** + - Rejected because it pushes raw exposure into user code and makes the safe path noisier than the risky path. + +## Drawbacks + +- Secret wrappers add friction when code genuinely needs raw strings or bytes. +- Redaction can create a false sense of security if users interpret it as encryption, access control, or memory-forensics protection. +- Encrypted idle storage has key-management and performance costs, and it cannot protect against every same-process threat. +- Equality, hashing, and serialization need conservative choices that may surprise users expecting string-like behavior. +- Stdlib modules and tooling must consistently honor the secret contract or the abstraction becomes unreliable. +- The exact reveal API needs careful design because it becomes the standard searchable marker for sensitive exposure. + +## Implementation architecture + +*(Non-normative.)* The Rust-backed implementation should use owned storage with redacting display and debug implementations. Where practical, secret payloads should be stored encrypted while idle with process-local key material and decrypted only inside scoped reveal guards. Plaintext buffers created by reveal guards should be zeroized when the guard closes. `SecretBytes` should use a zeroizing buffer where available. `SecretStr` may store UTF-8 in a protected byte buffer with fallible UTF-8 views, or use another representation that preserves the public contract. Stdlib consumers should pass secret wrappers through typed APIs and reveal internally only at the final trusted boundary. + +## Layers affected + +- **Stdlib / Runtime (`incan_stdlib`)**: must provide `std.secrets`, `SecretStr`, `SecretBytes`, redaction behavior, construction helpers, scoped reveal operations, protected-storage behavior where supported, and integration hooks for stdlib consumers. +- **Typechecker / Symbol resolution**: must preserve the distinct types and reject implicit conversion from secret wrappers to plain `str` or `bytes`. +- **Emission**: generated Rust must preserve redacting display/debug behavior and best-effort zeroization where promised. +- **Formatter**: no syntax changes are required, but examples and generated code should preserve readable secret-type annotations. +- **LSP / Tooling**: hover, completion, diagnostics, semantic inspection, action metadata, generated docs, and policy checks should preserve sensitivity metadata and make reveal operations discoverable. +- **Docs / Examples**: environment, CLI, HTTP, logging, telemetry, and workflow examples should demonstrate secret values instead of plain string tokens. + +## Unresolved questions + +- What are the exact reveal method names for `SecretStr` and `SecretBytes`? +- Should reveal operations return borrowed views, owned copies, scoped guards, or multiple variants? +- Should scoped reveal be the only stable v1 reveal surface, with owned plaintext extraction left for a later explicit escape hatch? +- Should encrypted idle storage be mandatory for all v1 targets, or a documented target capability with redaction and zeroization as the portable floor? +- How should process-local encryption keys be generated, stored, rotated, and destroyed? +- Should ordinary equality be available, or should secret comparison require explicit constant-time helper functions? +- Should `SecretStr` attempt to provide the same zeroization behavior as `SecretBytes`, or should the docs make `SecretStr` strictly a redaction-first wrapper? +- What exact redaction placeholder should display, debug, and diagnostic serialization use? +- Should default data serialization of secrets fail everywhere, or should some stdlib-owned formats serialize structured redaction objects by default? +- Should `std.secrets` eventually expose a generic `Secret[T]`, and if so, what protocol must `T` satisfy? +- Should secret provenance metadata distinguish environment variables, CLI input, config files, secret providers, and generated values in the initial surface? +- How should reveal sites interact with future policy approval, sandboxing, and capability checks? +- Should secret values participate in model field metadata automatically, or should fields still require an explicit `secret=true` marker for generated schema and docs? + + diff --git a/workspaces/docs-site/docs/RFCs/104_ambient_runtime_capabilities_and_receipts.md b/workspaces/docs-site/docs/RFCs/104_ambient_runtime_capabilities_and_receipts.md new file mode 100644 index 000000000..57d470d3d --- /dev/null +++ b/workspaces/docs-site/docs/RFCs/104_ambient_runtime_capabilities_and_receipts.md @@ -0,0 +1,445 @@ +# RFC 104: Ambient Runtime Capabilities and Receipts + +- **Status:** Draft +- **Created:** 2026-05-24 +- **Author(s):** Danny Meijer (@dannymeijer) +- **Related:** + - RFC 033 (`ctx` typed configuration context) + - RFC 055 (`std.fs` path-centric filesystem APIs) + - RFC 063 (`std.process` process spawning and command execution) + - RFC 066 (`std.http` HTTP client surface) + - RFC 075 (starter profiles and capability packs) + - RFC 076 (project mutation policy and recovery) + - RFC 078 (tool execution and typed workflow actions) + - RFC 089 (`std.environ` runtime environment access) + - RFC 090 (typed CLI framework) + - RFC 092 (interactive runtime stdlib contracts) + - RFC 093 (`std.telemetry`) + - RFC 094 (context managers) + - RFC 095 (`span` vocabulary blocks) + - RFC 102 (semantic layer inspection surface) + - RFC 103 (secret values and redaction-safe values) +- **Issue:** https://github.com/dannys-code-corner/incan/issues/662 +- **RFC PR:** - +- **Written against:** v0.3 +- **Shipped in:** — + +## Summary + +This RFC defines ambient runtime capabilities and receipts for Incan. Importing a module remains Python-readable and low ceremony, but using authority-bearing operations such as filesystem, environment, process, HTTP, clock, random, model, tool, or package-defined domain operations produces structured receipts and may be denied by a governed runtime. The stdlib is the first capability publisher, not the only one: library authors can define domain capabilities, attach receipt schemas, and participate in the same audit and policy system without reimplementing tracing or reaching for stdlib internals. The goal is ambient observation with explicit authority. + +## Core model + +Read this RFC as ten foundations: + +1. **Import is not authority:** source code may import `std.fs`, `std.process`, `std.environ`, `std.http`, or a capability-aware package without automatically receiving permission to perform those operations. +2. **Observation is ambient:** ordinary stdlib and library calls can emit structured receipts without requiring users to annotate every function with effect types. +3. **Authority is granted at boundaries:** runs, actions, tests, packages, and hosts grant capabilities; library code may request or declare capabilities, but cannot grant itself authority. +4. **Stdlib capabilities are built in:** host authority such as filesystem, environment, process, network, clock, random, model invocation, and tool invocation has reserved capability identities. +5. **Library capabilities are first-class:** packages may publish domain capabilities such as `example.policy.evaluate` or `example.index.query` that describe domain authority and receipt semantics. +6. **Receipts are not logs:** receipts are structured runtime facts with stable kinds, source spans where available, redaction state, status, and replay information; terminal logs are only one possible view. +7. **Strict enforcement is optional:** ordinary runs should remain simple, while governed runs can deny operations not covered by granted capabilities. +8. **Redaction is mandatory:** receipts must preserve sensitivity metadata and must not expose raw secret or policy-sensitive values by default. +9. **Replay claims must be honest:** the runtime should describe what can be replayed exactly, what requires fixtures, and what cannot be replayed. +10. **Policy consumes receipts:** policy systems, CI, editors, docs tooling, and agents consume the same capability declarations and receipt facts; they do not infer authority from prose or hidden conventions. + +## Motivation + +Python-shaped source is a major Incan strength, but Python's module model also hides authority. If Python code can import `os`, it can generally attempt to read environment variables, inspect and mutate files, spawn processes, or discover host state. External sandboxing can restrict that, but the source/module surface does not make authority visible or explainable. + +Incan should preserve the ergonomic part and reject the hidden-authority part. A user should be able to write ordinary readable code, import the modules they need, and run the program normally. When the same code is run in a governed context, the runtime should be able to say that a filesystem read, environment read, process spawn, HTTP request, model invocation, or package-defined domain operation was allowed, denied, redacted, or replay-limited. + +This matters most for real tools, automation, generated artifacts, policy-bound workflows, and agent-assisted maintenance. A failed or suspicious run should produce receipts that answer what authority was requested, what authority was granted, what actually happened, which values were redacted, which artifacts were touched, and what can be replayed. Without a shared capability and receipt model, every stdlib module and library will invent its own logs, policy hooks, and audit JSON. + +The key design constraint is usability. This RFC must not turn ordinary Incan into an algebraic-effect language where every helper function has capability algebra in its type signature. The default user experience should be: write normal Incan; capability-aware boundaries produce structured receipts; governed entrypoints can restrict and audit those receipts. + +## Goals + +- Split module availability from runtime authority. +- Define reserved host capability identities for common authority-bearing operations. +- Allow library authors to define domain capabilities and receipt schemas. +- Define ambient receipt emission for stdlib and library boundaries. +- Define governed runtime behavior when an operation requires a capability that was not granted. +- Define machine-readable run reports that include requested capabilities, granted capabilities, denied operations, emitted receipts, redaction state, and replay limits. +- Define how domain capabilities may imply or request host capabilities without granting themselves authority. +- Make receipts consumable by RFC 102 semantic inspection, RFC 078 typed actions, RFC 093 telemetry, RFC 076 policy, CI, LSP, docs tooling, and agents. +- Keep ordinary source readable and low ceremony. + +## Non-Goals + +- This RFC does not introduce a full algebraic effect system. +- This RFC does not require every function type to include a capability parameter or effect row. +- This RFC does not make imports fail merely because the current run has not granted a capability. +- This RFC does not define a complete operating-system sandbox. +- This RFC does not define no-std/freestanding target profiles, kernel support, unsafe/layout controls, panic strategy, or allocator strategy. Capability and receipt metadata may inform those later RFCs, but this RFC is not the freestanding/kernel RFC. +- This RFC does not guarantee perfect deterministic replay for external systems. +- This RFC does not replace `std.telemetry`, `std.logging`, diagnostics, or semantic inspection. +- This RFC does not require every package to publish capability metadata. +- This RFC does not allow libraries to grant themselves host authority. +- This RFC does not define the final CLI flag spelling for governed runs or reports. +- This RFC does not define a secret-value type; it only requires receipts to preserve sensitivity and redaction metadata from the owning subsystem. + +## Guide-level explanation + +Ordinary code should stay ordinary: + +```incan +from std.environ import env +from std.http import get + +def fetch_status() -> int: + url = env.get("STATUS_URL")? + response = get(url)? + return response.status.code +``` + +A normal run may behave just like a normal program: + +```text +incan run status.incn +``` + +An observed run asks the runtime to emit a machine-readable report: + +```text +incan run status.incn --report json +``` + +The report can show the authority-bearing operations that happened: + +```json +{ + "entrypoint": "status.fetch_status", + "granted_capabilities": [], + "mode": "observe", + "receipts": [ + { + "capability": "host.env.read", + "operation": "env.get", + "status": "observed", + "attributes": {"key": "STATUS_URL"}, + "redacted": false + }, + { + "capability": "host.http.request", + "operation": "http.request", + "status": "observed", + "attributes": {"method": "GET", "url_policy": "external", "status_code": 200}, + "redacted": false + } + ] +} +``` + +A governed run grants only selected authority: + +```text +incan run status.incn --allow host.env.read,host.http.request --report json +``` + +If the program later tries to spawn a process, the runtime should fail with a useful diagnostic: + +```text +status.incn:8 used std.process.Command.run(...) +This requires capability: host.process.spawn + +Granted capabilities: + host.env.read + host.http.request +``` + +Library authors should be able to participate without depending on stdlib-private hooks. A package can define a domain capability: + +```incan +capability example.policy.evaluate: + description = "Evaluate an input against a policy" + emits = "policy.evaluation" +``` + +The exact declaration syntax is unresolved. The important contract is that packages can publish stable capability identities, descriptions, receipt schemas, and relationships to host capabilities. + +Library code can then emit a receipt through a low-ceremony boundary: + +```incan +from std.runtime import receipts + +def evaluate(policy: Policy, input: Input) -> Decision: + with receipts.event("example.policy.evaluate", subject=policy.id): + return policy.evaluate(input) +``` + +For common entrypoints, typed actions can declare the capabilities they require: + +```incan +@action(caps=["example.policy.evaluate", "host.model.invoke"]) +def review(input: ReviewInput) -> ReviewReport: + ... +``` + +Granting a domain capability does not automatically let a package bypass host policy. If `example.policy.evaluate` needs `host.fs.read` to load a policy file, that relationship must be visible in metadata and accepted by the runtime or host policy. Libraries can name and explain authority; the runtime grants authority. + +## Reference-level explanation + +### Capability identities + +A capability identity must be a stable string. The exact naming grammar is unresolved, but this RFC reserves the `host.*` namespace for host authority capabilities owned by the Incan toolchain and runtime. + +Initial host capability families should include: + +- `host.env.read` +- `host.fs.read` +- `host.fs.write` +- `host.process.spawn` +- `host.http.request` +- `host.clock.read` +- `host.random` +- `host.model.invoke` +- `host.tool.invoke` + +Implementations may define narrower capabilities such as scoped filesystem paths, hostnames, methods, or model families, but the broad families must remain understandable in diagnostics and reports. + +Package-defined capabilities must be namespaced so two packages cannot accidentally define the same authority name. Package-defined capabilities may describe domain operations, typed actions, generated artifacts, policy checks, workflow steps, or library-specific effects. + +### Import, request, grant, and use + +Importing a module must not grant authority. Importing `std.process` is allowed even in a run that has not granted `host.process.spawn`. Authority is checked when an authority-bearing operation is invoked. + +A package, action, function, descriptor, or runtime operation may request capabilities. A run, host, action invoker, test harness, package manager, CI environment, or policy system may grant capabilities. Only the runtime or host authority boundary may decide whether a request is granted. + +When an operation requiring a capability is invoked in governed mode and the capability is not granted, the operation must fail before performing the authority-bearing behavior. The diagnostic must identify the required capability and should include the source span, import/module/function path, and a suggested grant spelling when available. + +### Runtime modes + +The runtime should support at least these conceptual modes: + +- `permissive`: operations run normally and receipts may be disabled. +- `observe`: operations run normally and receipts are emitted. +- `governed`: operations require granted capabilities and receipts are emitted. + +The exact CLI spelling is not normative. A natural user-facing shape is: + +```text +incan run app.incn --report json +incan run app.incn --allow host.env.read,host.http.request --report json +``` + +The default mode for ordinary local development is unresolved. The default must not surprise users by silently exporting data or sending reports to remote services. + +### Capability declarations + +A capability declaration should include: + +- stable identity; +- human-readable description; +- owning package or toolchain component; +- capability kind, such as host, library, action, artifact, or policy; +- optional implied or requested capabilities; +- optional scope schema, such as path, hostname, method, model, artifact kind, or action id; +- receipt event kinds emitted by the capability; +- redaction and sensitivity rules for receipt attributes; +- docs and diagnostic labels. + +Capability declarations may live in source, package metadata, manifest metadata, generated descriptors, or capability packs. Wherever they live, RFC 102 semantic inspection must be able to expose them as project facts. + +Package-defined capabilities must not grant host authority by implication alone. If a domain capability requests or implies `host.fs.read`, the runtime must resolve that relationship through host policy before allowing filesystem reads. + +### Receipts + +A receipt is a structured runtime fact emitted by a capability-aware operation. A receipt must include: + +- event id or sequence id; +- capability identity; +- operation kind; +- status, such as observed, allowed, denied, failed, redacted, or skipped; +- source location or semantic identity when available; +- package/module/function identity when available; +- parent span or context id when available; +- redacted attributes; +- sensitivity metadata; +- replay classification. + +A receipt should include operation-specific attributes such as environment variable key, filesystem path policy, HTTP method, URL policy, process command policy, model id policy, artifact id, action id, or policy id. Sensitive values must be redacted by default. + +Receipts must be machine-readable. Human output may summarize receipts, but human output must not be the integration contract. + +### Run reports + +A run report is a machine-readable summary of a run, action, test, or governed entrypoint. A report must include: + +- toolchain version; +- run mode; +- entrypoint identity; +- requested capabilities when available; +- granted capabilities; +- denied capability requests; +- emitted receipts; +- diagnostics; +- redaction summary; +- replay manifest or replay limitations. + +Reports may include artifact references, span trees, telemetry correlation ids, package versions, lockfile identity, source snapshot identity, and semantic package references. + +Reports must not include raw secret values or sensitive payloads unless a separate, explicit reveal policy approves that exposure. + +### Replay classification + +Each receipt and run report should classify replayability. Initial replay classifications should include: + +- `deterministic`: the operation can be replayed from recorded local inputs. +- `fixture-required`: replay requires recorded fixtures or test doubles. +- `external`: replay depends on external systems and cannot be exact without a recording. +- `unavailable`: replay is not supported for this operation. +- `redacted`: replay data exists but is intentionally hidden or incomplete. + +This RFC does not require the runtime to implement full replay. It requires the runtime to avoid dishonest replay claims. + +### Budgets + +Capability grants may include budgets. Budgets are optional constraints over granted authority, such as maximum request count, maximum bytes written, allowed path roots, allowed hosts, allowed process names, timeout limits, model-token limits, or artifact count. + +If a budget is exhausted in governed mode, the runtime must deny the operation before performing it where practical and must emit a denial receipt. If the operation cannot be prevented before partial work occurs, the receipt must describe the partial state honestly. + +### Library participation + +Library authors may define capabilities and receipt schemas. Libraries should not need to import stdlib-private modules or manually construct the full run report. + +The stdlib should provide a small public runtime receipt surface for library authors. The exact spelling is unresolved, but it should support scoped events, one-shot events, status updates, redacted attributes, and parent span/context attachment. + +Library-defined receipts must flow into the same run report as stdlib receipts. A package manager, LSP, CI job, or agent must not need separate integration logic for each library's audit output. + +### Relationship to telemetry + +Receipts and telemetry are related but distinct. Receipts are capability and authority facts. Telemetry is observability data. A receipt may be exported as a telemetry event or span attribute when telemetry is configured, but receipt generation must not require telemetry export. + +Receipts must remain available to local reports and policy systems even when `std.telemetry` is not configured. + +### Relationship to semantic inspection + +RFC 102 semantic inspection should expose declared capabilities, receipt schemas, action capability requirements, policy relationships, and report artifacts. Semantic inspection should not need to execute a program to discover static capability declarations. + +Runtime receipts may reference semantic identities from RFC 102 so tools can connect a run event back to source declarations, actions, generated artifacts, package metadata, and policy decisions. + +### Relationship to stdlib modules + +Stdlib modules that cross host authority boundaries must emit receipts when reporting is enabled and must enforce grants in governed mode. + +At minimum: + +- `std.environ` reads require `host.env.read`. +- `std.fs` reads require `host.fs.read`. +- `std.fs` writes require `host.fs.write`. +- `std.process` spawning requires `host.process.spawn`. +- `std.http` requests require `host.http.request`. +- clock APIs that read current time require `host.clock.read`. +- random APIs require `host.random`. +- model or tool invocation APIs require `host.model.invoke` or `host.tool.invoke`. + +Pure computation, parsing, formatting, local model construction, and in-memory transformations should not require host capabilities. + +## Design details + +### Syntax + +This RFC intentionally does not require new syntax. Capability declarations may eventually use source syntax, declaration metadata, package metadata, or manifest descriptors. The required contract is capability identity, declaration, grant, enforcement, receipt emission, and inspection. + +Illustrative source syntax such as `capability example.policy.evaluate:` is non-normative. + +### Semantics + +Capability checks occur at authority-bearing operation boundaries. In ordinary source, a helper function that calls `std.http.get` does not need to declare an effect type merely because it may perform HTTP. If the program runs in governed mode without `host.http.request`, the operation fails at the boundary with a capability diagnostic. + +Static capability declarations are still useful for actions, packages, generated artifacts, docs, and policy review. They should describe expected authority before a run happens. Runtime receipts describe actual authority use during a run. + +Static declarations and runtime receipts should be compared where possible. If a run uses a capability not declared by its action or package metadata, the report should mark that mismatch. + +### Interaction with existing features + +- **RFC 033 (`ctx`)**: configuration fields may require environment or secret-provider capabilities when resolved at runtime. +- **RFC 055 (`std.fs`)**: file APIs become standard publishers of filesystem receipts and governed checks. +- **RFC 063 (`std.process`)**: process spawning becomes a governed host capability with structured command-policy receipts. +- **RFC 066 (`std.http`)**: HTTP requests become governed host capabilities with redacted request/response receipts and replay classifications. +- **RFC 075 (capability packs)**: project capability packs may declare expected package and action capabilities, but they must not grant host authority without runtime policy. +- **RFC 076 (policy)**: policy consumes capability declarations and receipts, and may approve, deny, or require review for grants and mutations. +- **RFC 078 (typed actions)**: actions may declare required capabilities and emit action-scoped reports. +- **RFC 089 (`std.environ`)**: environment access becomes a governed and receipted host boundary. +- **RFC 090 (typed CLI framework)**: CLI commands may declare capability requirements and expose helpful denial diagnostics. +- **RFC 092 (interactive runtime contracts)**: target manifests may describe host capabilities supported by a runtime target. +- **RFC 093 (`std.telemetry`)**: telemetry may export receipts, but receipts remain local authority facts when telemetry is disabled. +- **RFC 094 and RFC 095**: context managers and span vocabulary blocks provide convenient scopes for receipt correlation, but receipts do not require span syntax. +- **RFC 102 (semantic inspection)**: capability declarations, receipt schemas, run reports, and replay manifests become inspectable semantic artifacts. +- **RFC 103 (secret values)**: receipt redaction should preserve secret-value sensitivity metadata without requiring receipts to expose raw secret payloads. + +### Compatibility + +This RFC is additive. Existing programs can continue to run in permissive mode. Governed mode may reveal hidden authority assumptions in existing programs, but those failures are the point of governed execution and must be diagnosable. + +Stdlib APIs that already perform authority-bearing operations should be updated to emit receipts and enforce grants in governed mode. Libraries may opt in incrementally by publishing capability descriptors and using the public receipt surface. + +## Alternatives considered + +### Full algebraic effects + +Rejected for now. Algebraic effects or effect rows may become useful later, but they would fight Incan's Python-shaped ergonomics if introduced as the first user-facing authority model. + +### Stdlib-only auditing + +Rejected because it would prevent library authors from defining domain capabilities and would force every serious package to invent its own audit layer. + +### External sandbox only + +Rejected because external sandboxing can restrict behavior but does not provide source-level capability identities, semantic inspection, domain receipts, or useful diagnostics. + +### Logging-only receipts + +Rejected because logs are human-oriented and often unstructured. Receipts must be machine-readable authority facts with stable semantics, redaction, and replay information. + +### Import-time capability checks + +Rejected because it makes code harder to reuse and breaks ordinary Python-shaped authoring. Authority should be checked when authority-bearing operations are invoked, not when modules are imported. + +## Drawbacks + +This RFC adds a cross-cutting runtime contract. Stdlib modules, package metadata, typed actions, policy, LSP, reports, and agents must agree on capability identities and receipt shapes. + +Capability names can sprawl if packages publish overly fine-grained or inconsistent capability vocabularies. Tooling will need naming guidance, validation, and docs support. + +Receipts can create overhead and sensitive metadata risk. Implementations must make reporting configurable, preserve redaction, and avoid accidental remote export. + +Governed mode can frustrate users if diagnostics are vague or if common operations require too many grants. The initial capability set should stay coarse and understandable until real usage proves finer scope is needed. + +## Implementation architecture + +This section is non-normative. + +A practical architecture is to route capability-aware operations through a runtime authority context. That context can hold run mode, grants, budgets, redaction policy, receipt sink, telemetry bridge, and source/semantic identity mapping. + +Stdlib modules should call a small shared runtime authority API before crossing host boundaries and emit receipts through the same API after success, failure, denial, or partial completion. Library authors should get a public receipt API that creates domain receipts without exposing private stdlib internals. + +Generated build artifacts and run reports should be ordinary artifacts that RFC 102 can inspect. LSP, CI, docs tooling, and agents should consume the report schema rather than parsing logs. + +## Layers affected + +- **Stdlib / Runtime (`incan_stdlib`)**: host-boundary modules need capability checks, receipt emission, redaction handling, and report integration. +- **Tooling / CLI**: run, test, action, and build commands need report output, governed-mode grants, denial diagnostics, and machine-readable schemas. +- **Package metadata**: packages need a way to publish capability declarations and receipt schemas. +- **Typechecker / Semantic metadata**: static capability declarations and action requirements should be exposed as checked metadata where available. +- **IR Lowering / Backend**: source spans and semantic identities should be preserved well enough for receipts to point back to source and semantic objects. +- **LSP / Docs tooling**: editors and docs can surface capability declarations, required grants, denial diagnostics, and report artifacts. +- **Policy / CI / Agents**: policy and automation can consume capability declarations and receipts to decide whether runs, actions, generated artifacts, or proposed changes are acceptable. + +## Unresolved questions + +- What is the exact grammar for capability identities? +- Should capability declarations live in source syntax, declaration metadata, package manifests, or all of them? +- What should the default run mode be for `incan run`, `incan test`, and typed actions? +- What is the minimum stable host capability set? +- How should scoped grants be represented for paths, hosts, methods, models, tools, and artifacts? +- Should package-defined capabilities be allowed to imply host capabilities automatically when a user grants the package capability, or should host grants always be listed separately? +- What is the first stable receipt schema version? +- How should receipt sinks be configured, and where should reports be written by default? +- Which replay classifications are required for the first implementation? +- How should telemetry export represent receipts without making telemetry a dependency of receipt generation? +- How should capability budgets be expressed in CLI, package metadata, and typed action declarations? + + diff --git a/workspaces/docs-site/docs/RFCs/105_architect_rule_engine.md b/workspaces/docs-site/docs/RFCs/105_architect_rule_engine.md new file mode 100644 index 000000000..b52b59486 --- /dev/null +++ b/workspaces/docs-site/docs/RFCs/105_architect_rule_engine.md @@ -0,0 +1,373 @@ +# RFC 105: `incan architect` rule engine for design, safety, idiom, and smell findings + +- **Status:** Draft +- **Created:** 2026-05-24 +- **Author(s):** Danny Meijer (@dannymeijer) +- **Related:** + - RFC 006 (generators) + - RFC 048 (contract-backed models emit and tooling) + - RFC 070 (Result combinators) + - RFC 088 (iterator adapter surface) + - RFC 096 (declaration metadata blocks) +- **Issue:** https://github.com/dannys-code-corner/incan/issues/663 +- **RFC PR:** - +- **Written against:** v0.3 +- **Shipped in:** — + +## Summary + +This RFC proposes `incan architect` as a deterministic code-advice command for Incan projects. The command reports evidence-backed findings across architecture, safety, idiom usage, and maintainability smells by running maintainable rules over compiler-backed codegraph facts. The central goal is not to create a broad subjective linter, but to create a durable rule authoring surface where new advice can be added cheaply, tested precisely, calibrated against real projects, and consumed by humans, agents, editors, and CI without relying on model inference for core detection. + +## Core model + +1. **Compiler-backed facts first:** `incan architect` consumes source facts produced by Incan's parser, module/import resolver, typechecker, metadata pipeline, and codegraph exporter rather than independently scraping text. +2. **Rules interpret facts:** Each rule consumes typed fact views and emits findings with stable codes, priorities, categories, confidence, evidence, suggestions, and risks. +3. **Findings are advisory:** Architect findings are not compiler errors. They describe design pressure or code-shape opportunities with enough evidence for a human or agent to decide whether to act. +4. **Categories are explicit:** Architecture findings, safety findings, idiom findings, and code-smell findings remain separate in rule codes and profiles even when they share one command. +5. **Conservative detection is preferred:** The command should under-report ambiguous style opportunities rather than produce noisy, low-trust advice. +6. **Rule authoring is a product surface:** The feature is only maintainable if adding a rule means using stable typed facts and reusable queries, not hand-parsing raw graph nodes or reimplementing AST walks. + +## Motivation + +Incan already has syntax checks, semantic checks, formatter behavior, tests, and generated-Rust validation. Those tools answer whether a program parses, typechecks, formats, and runs. They do not answer whether a project is accumulating design pressure: repeated dispatch over the same domain, public boundaries that can panic on recoverable input, old-shaped control flow that should now use language features, or small helper functions that add indirection without carrying domain meaning. + +The first experiments with an architecture-advice command showed that deterministic rules can surface useful pressure when they report concrete source evidence and stay cautious about severity. Repeated match dispatch can reveal a growing operation boundary. Fail-fast calls inside public APIs can reveal recoverability problems. Body-shape facts can also support smaller maintainability smells such as compound-assignment candidates, single-use trivial helpers, append-only list builders that could become comprehensions, or `Result` matches that could use RFC 070 combinators. + +Without a formal rule engine, each new check risks becoming a one-off command-private AST walk with custom parsing, inconsistent output, and ad hoc severity. That path does not scale. The value is in a shared substrate: one project-wide codegraph, one typed query layer, one finding model, one de-duplication path, and many small rules that are easy to review and calibrate. + +This feature also matters for agent workflows. Agents can already make broad refactoring suggestions, but those suggestions are often expensive to verify and easy to overfit. `incan architect` should provide deterministic evidence that an agent can use as grounding: exact files, lines, matched domains, shared patterns, call sites, usage counts, and counterexample risks. A model may later summarize or prioritize findings, but the core detection should remain inspectable and reproducible. + +## Goals + +- Define `incan architect` as the umbrella command for deterministic design, safety, idiom, and maintainability-smell advice. +- Provide a stable finding model with rule code, category, priority, confidence, evidence, pressure, suggestions, risks, and machine-readable output. +- Provide project-wide directory scanning over `.incn` source trees with deterministic module de-duplication and finding de-duplication. +- Establish rule categories and profiles so users can run architecture-only, safety-only, idiom-only, smell-only, or all-rule scans. +- Establish a maintainable rule authoring surface based on typed facts and reusable queries over codegraph data. +- Extend codegraph body facts as needed for rule families such as match dispatch, call sites, references, assignment/update shapes, helper usage, loop-builder shapes, and result-match shapes. +- Include code smells in scope when they can be detected conservatively with clear evidence and useful counterexamples. +- Keep detection deterministic for the first version; no language model is required for core finding generation. +- Support text output for humans and stable JSON output for tools, agents, editors, and CI. +- Make suppression and baselining part of the product model so mature codebases can adopt the command incrementally. + +## Non-Goals + +- This RFC does not make architect findings compiler errors. +- This RFC does not replace formatter rules, typechecker diagnostics, Clippy-style generated-Rust checks, or project tests. +- This RFC does not require a small language model or remote AI service for rule detection. +- This RFC does not attempt to infer developer intent from names alone. +- This RFC does not require every possible code smell to ship in the first version. +- This RFC does not define automatic rewrites or apply fixes. +- This RFC does not define a public plugin ABI for third-party binary rule packages. +- This RFC does not require every codegraph fact to be part of a permanently stable external schema in the first release; only the JSON findings format and documented command behavior need v0.5 stability. + +## Guide-level explanation + +Users run `incan architect` on a file or project directory. + +```bash +incan architect . +incan architect src/lib.incn --format json +incan architect . --profile architecture +incan architect . --profile smells +``` + +The command prints findings grouped by priority and grounded in source evidence. + +```text +[P3] Repeated match dispatch over `source_kind` +Pressure: 2 match expressions dispatch over `source_kind` and share 3/3 explicit arms: SourceKind.Arrow(...), SourceKind.Csv(...), SourceKind.Parquet(...) +Suggestions: + - Decide whether this is intentionally exhaustive local logic or a growing operation boundary. + - If it is a growing operation boundary, prefer an adapter or registry outside the domain type when the operation belongs to another subsystem. +Risks: + - Keep local exhaustive matches when they are clearer than an abstraction and the case set changes rarely. +Evidence: + - src/backend.incn:160:5 in register_one (explicit arms: 3/3; fallback: no) + - src/schema.incn:322:5 in schema_columns_for_source (explicit arms: 3/3; fallback: no) +``` + +The architecture value is not merely that two matches are textually similar. The useful signal is that separate subsystems are making parallel decisions over the same closed domain. For example, an ingestion package might register execution backends in one module and infer schemas in another module, with both operations matching every `SourceKind` variant. + +```incan +def register_backend(kind: SourceKind, registry: BackendRegistry) -> None: + match kind: + SourceKind.Csv(_) => registry.add("csv", csv_backend()) + SourceKind.Json(_) => registry.add("json", json_backend()) + SourceKind.Parquet(_) => registry.add("parquet", parquet_backend()) + + +def infer_columns(source: Source) -> Result[list[Column], SchemaError]: + match source.kind: + SourceKind.Csv(_) => return infer_csv_columns(source) + SourceKind.Json(_) => return infer_json_columns(source) + SourceKind.Parquet(_) => return infer_parquet_columns(source) +``` + +The recommendation should not be "put backend registration and schema inference methods on `SourceKind`." That would move subsystem responsibilities onto the enum. The more architectural advice is to ask whether this is a growing operation boundary. If every new source format requires coordinated edits to backend registration, schema inference, validation, documentation, and test fixtures, the code may want a format-handler registry or adapter table where each format owns its related operations. + +```text +[P3] Repeated match dispatch over `source.kind` +Pressure: backend registration and schema inference both dispatch over all source formats. +Suggestion: Consider a format-handler registry if adding one format requires shotgun edits across subsystems. +Risk: Keep exhaustive local matches if the format set is closed, the operations are genuinely local, and cross-format registration would obscure control flow. +``` + +Architect findings use categories. Architecture findings describe design pressure. Safety findings describe failure or recoverability risk. Idiom findings describe opportunities to use Incan features more directly. Smell findings describe local maintainability pressure. + +```text +safety.fail_fast_boundary_call +idiom.result_combinator_candidate +smell.single_use_trivial_helper +arch.repeated_match_dispatch +``` + +Small smells are allowed when they are precise and humble. A trivial helper rule can identify a private helper that is used once and only returns a pure expression. + +```incan +def add(left: int, right: int) -> int: + return left + right +``` + +The finding should not say that the helper is definitely wrong. It should say that the helper may be unnecessary unless its name carries useful domain meaning. + +```text +[P3] Private helper only wraps one expression +Pressure: `add` is private, used once, and only returns `left + right`. +Suggestion: Inline the expression if the helper does not name a useful domain concept. +Risk: Keep the helper if it documents intent, preserves API shape, acts as a callback, or is expected to grow. +``` + +A comprehension candidate should likewise report a specific body shape, not a broad preference. + +```incan +def positive_scores(scores: list[int]) -> list[int]: + out = [] + for score in scores: + if score > 0: + out.append(score) + return out +``` + +The corresponding advice is useful only because the shape is append-only, the accumulator is returned, and no other mutation or side effect participates in the loop. + +```text +[P3] Append-only list builder can be a comprehension +Pressure: `positive_scores` builds and returns a list with one append-only loop. +Suggestion: Use `[score for score in scores if score > 0]` if the eager list is the intended result. +Risk: Keep the loop if additional statements, logging, early exits, or mutation are part of the real workflow. +``` + +For RFC 070 `Result` combinators, architect can identify obvious match shapes and suggest the equivalent method only when the transformation is mechanically recognizable. + +```incan +match parsed: + Ok(value) => Ok(clean(value)) + Err(err) => Err(err) +``` + +The finding can suggest `parsed.map(clean)` because one branch transforms the `Ok` payload and the `Err` branch passes through unchanged. + +## Reference-level explanation + +### Command behavior + +`incan architect [PATH] [OPTIONS]` must accept a source file or directory. When `PATH` is omitted, the command should scan the current directory. + +When `PATH` is a file, the command must scan the file and the modules needed to resolve its imports according to ordinary Incan module rules. + +When `PATH` is a directory, the command must scan `.incn` files under that directory recursively. The scan must be deterministic. The scan must de-duplicate modules by source path so a file imported by multiple roots contributes facts once. + +The command must provide `--format text` and `--format json`. Text output is for humans. JSON output is the integration surface for agents, editors, CI, dashboards, and future baselining tools. + +The command should provide `--profile` with at least `architecture`, `safety`, `idioms`, `smells`, and `all`. The default profile is unresolved by this draft. + +### Finding model + +Every finding must have a stable rule code. Rule codes must be namespaced by category. + +```text +arch.repeated_match_dispatch +safety.fail_fast_boundary_call +idiom.result_combinator_candidate +smell.single_use_trivial_helper +``` + +Every finding must include a category, priority, confidence, title, pressure, evidence, suggestions, and risks. + +Priority must describe expected action pressure, not proof certainty. + +```text +P1: likely correctness, reliability, or public-boundary risk that should be reviewed before release +P2: meaningful design or maintainability pressure that should be tracked or scheduled +P3: watchlist, idiom, or local smell that may be worth cleanup when nearby work touches the code +Info: low-pressure educational or style-level advice +``` + +Confidence must describe how mechanically strong the rule match is. + +```text +High: the rule found a narrow, mechanically recognizable shape +Medium: the rule found a useful pattern with plausible counterexamples +Low: the rule is exploratory and should normally be hidden outside explicit profiles +``` + +Evidence must identify source file, line, column, owner declaration when available, and rule-specific context. Rule-specific context may include matched arms, overlap counts, fallback/default-arm presence, callee labels, usage counts, body-shape summaries, or suggested replacement text. + +Suggestions must be phrased as advice, not certainty. Risks must name the common counterexamples that would make the suggestion wrong. + +Findings must be de-duplicated before output. Identical findings produced through multiple import roots must appear once. + +### Rule categories + +Architecture rules describe design pressure across declarations, modules, domains, or boundaries. Repeated match dispatch, growing literal domains, and operation-boundary pressure belong here. + +Safety rules describe recoverability, fail-fast behavior, partial handling, unchecked assumptions, or public-boundary hazards. A public function that can panic on caller-provided data belongs here. + +Idiom rules describe opportunities to use Incan language or stdlib features more directly. Result combinator candidates, iterator adapter candidates, generator/comprehension candidates, and compound assignment candidates belong here. + +Smell rules describe local maintainability pressure. Single-use trivial helpers, repeated literals, unnecessary wrappers, long branch-heavy functions, and append-only builders belong here when detected conservatively. + +Rules must not be categorized as architecture findings merely because they are emitted by `incan architect`. + +### Rule authoring contract + +Rules must declare metadata: code, category, default priority, default confidence, profile membership, required fact kinds, and a short explanation. + +Rules must consume typed fact views rather than raw serialized facts. A rule that needs match dispatch sites, call sites, assignment shapes, helper usage counts, or loop-builder shapes should ask for those views directly. + +Rules should be small and independently testable. Each rule should have positive and negative fixtures. Negative fixtures are required for common counterexamples named in the rule's risk text. + +Rules must not require typechecked metadata when a syntactic fact is sufficient. Rules may use type facts when precision depends on type information, such as recognizing `Result[T, E]` match shapes. + +Rules should prefer narrow body-shape facts over broad textual heuristics. For example, a comprehension candidate should be based on an append-only list-builder shape, not the mere presence of a `for` loop and `append`. + +Rules must not emit findings for generated stdlib internals or known external code unless the user explicitly scans those sources. + +### Codegraph fact requirements + +The codegraph exporter must provide enough source facts for rules to avoid command-private AST walks. The first useful fact families are declarations, imports, public API metadata, match dispatches, call sites, references, assignment/update shapes, function body summaries, usage counts, loop-builder shapes, and result-match shapes. + +Match dispatch facts must include the matched domain, explicit pattern labels, explicit pattern count, source arm count, and wildcard/default-arm context. + +Call-site facts must include callee key, callee label, receiver shape when available, source location, and owner declaration. + +Reference facts must support usage counting for private declarations and helper functions. + +Assignment/update facts must make compound-assignment candidates expressible without string matching. + +Function body summary facts should identify simple shapes such as single-return expression, pure expression wrapper, append-only list builder, and short result-match transform. These summaries must be conservative. + +Result-match facts should identify branch-preserving transformations only when the matched expression is known to be a `Result[T, E]` or the syntactic shape is unambiguous enough for an idiom finding with appropriate confidence. + +### Suppression and baselining + +The command should support local suppression of a specific rule at a specific source location. Suppression syntax is unresolved by this draft. + +The command should support project baselines so existing findings can be recorded and new findings can fail CI or be highlighted separately. Baseline storage is unresolved by this draft. + +Suppressions and baselines must preserve rule code and evidence identity. A future change that moves or changes the evidence should not silently suppress an unrelated finding. + +## Design details + +### Profiles + +Profiles let users choose the kind of advice they want. `architecture` should include cross-cutting design pressure. `safety` should include fail-fast and recoverability risk. `idioms` should include feature-usage opportunities. `smells` should include local maintainability findings. `all` should include every non-experimental rule. + +Rules may belong to more than one profile only when that does not blur the category. For example, a public fail-fast boundary call is a safety finding even if it also has architecture implications. + +Exploratory rules may exist behind an explicit experimental profile, but they must not be enabled by default. + +### Severity calibration + +Severity should be calibrated against evidence strength, public surface impact, and likely cost of ignoring the finding. Public API failures are generally higher priority than private helper smells. Repeated design pressure across files is generally higher priority than a local expression-level cleanup. Idiom suggestions are generally P3 or Info unless the shape creates repeated complexity or risk. + +Rules should downrank or suppress known low-action cases. For example, fail-fast calls around trusted constants may be lower priority than fail-fast calls around caller-provided input. Exhaustive matches over a closed domain may be preferable to abstraction when the matched operation is local and the domain changes rarely. + +### Examples of initial rules + +`arch.repeated_match_dispatch` reports repeated match expressions that dispatch over the same domain and share multiple explicit arms. The rule should report overlap counts and wildcard/default context. + +`safety.fail_fast_boundary_call` reports `unwrap`, `expect`, `panic`, `todo`, and `unreachable` inside public or internal boundaries. Public API boundaries should generally be P1. Internal boundaries should generally be P2 unless evidence shows trusted constants or invariant setup. + +`idiom.result_combinator_candidate` reports obvious RFC 070 match shapes that can be expressed with `map`, `map_err`, `and_then`, `or_else`, `inspect`, or `inspect_err`. + +`idiom.compound_assignment_candidate` reports assignments such as `i = i + 1` when the target and left operand are the same simple storage place and `i += 1` is equivalent. + +`idiom.comprehension_candidate` reports append-only list builders that can be represented as eager list comprehensions. + +`smell.single_use_trivial_helper` reports private, undocumented, undecorated helpers that are used once and only return a simple pure expression. The rule must mention that domain vocabulary can justify keeping the helper. + +`smell.repeated_literal_domain` reports repeated raw string or scalar literal domains used as branch keys or dispatch keys across multiple sites. + +## Alternatives considered + +### Keep architect as architecture-only + +This would preserve a narrow name, but it would force closely related idiom and smell findings into a separate command even though they need the same project-wide codegraph, evidence model, de-duplication, profiles, suppressions, and JSON output. The better boundary is category namespace, not separate infrastructure. + +### Build a general linter instead + +A general linter would fit small syntax-level advice, but it would understate the project-wide design-pressure use case. The command should remain broader than a linter while still identifying local smells as one category. + +### Use a language model for rule detection + +Model-based detection may be useful later for summarization, clustering, or explaining findings in pull requests. It is not the right foundation for v0.5 rule detection because findings need to be reproducible, testable, source-grounded, and suitable for CI. + +### Let every rule walk the AST directly + +This is the fastest way to add a first rule and the worst way to maintain many rules. It duplicates traversal logic, fragments fact extraction, and makes rule behavior harder to share with agents, editors, and other code-intelligence tools. + +### Make findings auto-fixable from the start + +Some findings will eventually support safe rewrites, such as compound assignment candidates. Making fixes part of the first version would expand the scope into formatter, semantic preservation, and edit application. The first version should focus on reliable findings and stable output. + +## Drawbacks + +This feature adds a new advisory surface that can become noisy if rule quality is poor. The command must earn trust by being conservative, showing evidence, and naming counterexamples. + +The codegraph fact model will grow. If facts are added without a typed query layer, rules will become stringly and brittle. If facts are over-designed too early, implementation will slow down before the rule set proves itself. + +Some code smells are subjective. A helper that looks unnecessary may carry important domain meaning. A loop that could be a comprehension may be clearer as a loop when side effects are about to be added. The finding model must make room for this uncertainty through confidence and risk text. + +Project-wide scanning may be slower than entry-point scanning. The implementation should keep scans deterministic and should leave room for caching, but v0.5 should prioritize correctness and evidence over premature optimization. + +## Implementation architecture + +This section is non-normative. + +The recommended internal shape is a layered pipeline: source collection, compiler-backed codegraph extraction, typed fact views, query indexes, independent rule modules, finding normalization, de-duplication, profile filtering, and text/JSON rendering. + +The codegraph layer should remain the producer of source facts. The architect layer should not own parsing or typechecking behavior. Architect rules should operate over typed views such as match dispatch sites, call sites, references, assignment/update candidates, usage counts, loop-builder shapes, and result-match shapes. + +The rule engine should provide a small metadata contract for rule authors. A rule should declare its code, category, default priority, confidence, profiles, required facts, and explanation. A rule should receive a query context and emit findings. + +The report layer should be shared by all rules. Sorting, de-duplication, JSON serialization, text formatting, suppression matching, and baseline matching should not be implemented per rule. + +The first version should ship with a small calibrated rule set rather than a large catalogue. New rules should be added only when they have clear positive fixtures, negative fixtures, and calibration evidence from real source. + +## Layers affected + +- **Parser / AST**: No new user syntax is required, but source traversal must expose enough body shapes for codegraph facts. +- **Typechecker / Symbol resolution**: Rules may need checked public API metadata, resolved imports, type facts for `Result` shapes, and symbol usage information. +- **IR Lowering**: No required impact. +- **Emission**: No required impact. +- **Stdlib / Runtime (`incan_stdlib`)**: No required runtime impact, though stdlib feature surfaces such as Result combinators and iterator adapters inform idiom rules. +- **Formatter**: No required impact unless future auto-fix support is added. +- **LSP / Tooling**: The JSON findings format should be usable by editors, agents, CI, and future diagnostics-style surfaces. +- **CLI / Project tooling**: `incan architect` needs project-wide scanning, profiles, stable text/JSON output, suppression support, and baseline support. +- **Documentation**: The CLI reference must document command behavior, profiles, categories, priorities, confidence, suppressions, and examples. + +## Unresolved questions + +- What is the default profile for `incan architect .`: architecture-only, architecture plus safety, or all stable rules? +- What suppression syntax should Incan use for architect findings, and should it share vocabulary with compiler diagnostic suppressions? +- Should baselines live in `incan.toml`, a separate lock-like file, or a generated artifact under project tooling state? +- Which finding fields are stable enough to commit as v0.5 JSON output, and which should remain experimental? +- Should code-smell findings use the namespace `smell.*` or `maintainability.*`? +- Should project-wide directory scanning include tests by default, and should findings from tests use a separate priority calibration? +- How should architect distinguish trusted-constant fail-fast calls from caller-input fail-fast calls in a deterministic, maintainable way? +- Should third-party rule packages be considered after v0.5, or should v0.5 explicitly restrict rule authoring to the Incan repository? + + diff --git a/workspaces/docs-site/docs/RFCs/closed/implemented/036_user_defined_decorators.md b/workspaces/docs-site/docs/RFCs/closed/implemented/036_user_defined_decorators.md index 6ab95efcf..7ed90b70d 100644 --- a/workspaces/docs-site/docs/RFCs/closed/implemented/036_user_defined_decorators.md +++ b/workspaces/docs-site/docs/RFCs/closed/implemented/036_user_defined_decorators.md @@ -13,7 +13,7 @@ - RFC 031 (Library system — enables decorator libraries to ship as `pub::` packages) - RFC 037 (Native web and HTTP stdlib redesign — consumer of `@app.get` / `@app.post`) - RFC 084 (RHS partial callable presets — future decorator factory ergonomics) -- **Issue:** [#170](https://github.com/dannys-code-corner/incan/issues/170) +- **Issue:** [#170](https://github.com/dannys-code-corner/incan/issues/170), [#640](https://github.com/dannys-code-corner/incan/issues/640) - **RFC PR:** — - **Written against:** v0.2 - **Shipped in:** v0.3 @@ -92,7 +92,7 @@ This desugars to the `@app.get`/`@app.post` decorator form, which itself desugar - Desugar user-defined decorators to ordinary callable application before type checking. - Apply stacked decorators bottom-up, matching Python's decorator ordering. - Type-check decorator application through the ordinary callable and assignment rules. -- Allow decorator calls to change the visible type of the decorated binding. +- Allow decorator calls to change the visible callable type of the decorated binding. - Keep decorator semantics compile-time and declaration-oriented; the language must not introduce arbitrary module-level statement execution or module-initialization side effects for decorators. - Provide the primitive needed for library-owned patterns such as `@app.get`, `@cache`, `@retry`, and `@validate`. @@ -257,7 +257,7 @@ f = D2(f) f = D1(f) ``` -This means `D1` wraps `D2`'s result, which wraps `D3`'s result, which wraps the original `f`. Each step may change the type of `f`. +This means `D1` wraps `D2`'s result, which wraps `D3`'s result, which wraps the original `f`. Each step may change the callable type of `f`. **Scope of desugaring** — user-defined decorators desugar on `def`, `async def`, and method declarations. Class, model, trait, newtype, enum, field, alias, and module declarations are out of scope for this RFC. @@ -275,10 +275,35 @@ After desugaring, the typechecker treats `f = D(f)` as a regular call expression 1. `D` must be a callable. If it is not, the compiler emits `decorator 'D' is not callable`. 2. The argument type of `D`'s first parameter must be compatible with `f`'s declared type. -3. The return type of `D(f)` becomes the new type of `f` in the enclosing scope. If `D` returns the same function type it received, `f`'s type is unchanged. If the return type cannot be inferred, an explicit return type annotation on `D` is required. +3. The return type of `D(f)` must itself be callable and becomes the new callable type of `f` in the enclosing scope. If `D` returns the same function type it received, `f`'s type is unchanged. If the return type cannot be inferred, an explicit return type annotation on `D` is required. For decorator factories, step 1 applies to `D(args)` — the factory expression must produce a callable-shaped value — and then steps 2 and 3 apply to that callable applied to `f`. +### v0.3 amendment: generic decorator factories + +Issue #640 was accepted as an implementation amendment to this RFC because it naturally extends decorator factories rather than introducing a separate decorator model. A decorator factory may be generic over the decorated function type and return `((F) -> F)`, letting libraries write one registration helper instead of one helper per callable signature: + +```incan +pub def registered[F](function_ref: str) -> ((F) -> F): + return (func) => func + +@registered("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +The compiler infers `F` from the decorated function when applying the produced decorator. If inference needs an explicit call-site type, the decorator factory call accepts the same bracketed type-argument syntax as ordinary generic calls: + +```incan +@registered[(str) -> ColumnExpr]("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +This amendment preserves RFC 036's binding contract: later references, exports, imports, checked API metadata, and editor surfaces observe the concrete decorated function signature unless the decorator intentionally returns a different callable shape. + +Python decorators can replace a function binding with an arbitrary object. Incan intentionally does not copy that dynamic part of Python's model: user-defined function and method decorators are callable-to-callable transforms. Python's `Callable[[A, B], R]` corresponds to Incan's `(A, B) -> R`; `=>` is only for closure expressions, not callable types. The common generic registry shape is `(F) -> F`; wrappers that intentionally change the callable signature should spell both the source callable type and replacement callable type explicitly. + ### Async decorators A decorator applied to an `async def` receives an async function value. The decorator is responsible for preserving async semantics correctly — typically by defining an `async def wrapper(...)` internally. The compiler does not automatically lift a synchronous wrapper to async; a sync decorator applied to an async function produces a sync-typed result, which is likely a type error at the call site. @@ -296,7 +321,7 @@ A decorator applied to an `async def` receives an async function value. The deco ### Syntax -No new decorator syntax is introduced. `@name` and `@name(args)` already parse. Unknown decorator names no longer produce an error on `def`, `async def`, or method declarations — they desugar instead. +RFC 036 originally required no new decorator syntax beyond `@name` and `@name(args)`. The v0.3 implementation amendment also accepts explicit generic call-site arguments on decorator factory calls, as in `@name[T](args)`, using the same type-argument syntax as ordinary generic calls. Unknown decorator names no longer produce an error on `def`, `async def`, or method declarations — they desugar instead. Method decorator signatures use reference callable parameters for receivers. Immutable method receivers are written as `&Owner`, and mutable method receivers are written as `&mut Owner`, for example `(&Box, int) -> str` and `(&mut Counter, int) -> int`. diff --git a/workspaces/docs-site/docs/language/reference/feature_inventory.md b/workspaces/docs-site/docs/language/reference/feature_inventory.md index 063d94bae..355577557 100644 --- a/workspaces/docs-site/docs/language/reference/feature_inventory.md +++ b/workspaces/docs-site/docs/language/reference/feature_inventory.md @@ -39,7 +39,7 @@ Use it when deciding whether code should use an existing Incan surface before ad | Symbol, method, and variant aliases | Syntax | 0.3 | None. | `pub average = alias avg`
`mean = avg`
`WARNING = alias WARN` | Aliases expose another resolved name for the same declaration, method, or enum variant without duplicating behavior. | Wrapper functions or duplicated enum variants used only for compatibility names. | [Symbol aliases](symbol_aliases.md), [Imports and modules](imports_and_modules.md), [Release 0.3](../../release_notes/0_3.md) | | Callable presets with `partial` | Syntax | 0.3 | None. | `pub get = partial route(method="GET")`
`set_alive = partial set_state(state=true)` | `partial` creates a callable surface from an existing callable by supplying named preset values. | Hand-written wrappers whose only job is to pass the same keyword defaults. | [Callable presets](callable_presets.md), [Callable presets explained](../explanation/callable_presets.md), [Release 0.3](../../release_notes/0_3.md) | | Rest parameters, unpacking, and spreads | Syntax | 0.3 | None. | `def log(*items: str, **fields: str) -> None:`
`f(*xs, **kw)`
`[*prefix, item]`
`{**base, "x": 1}` | Functions can capture `*args` / `**kwargs`; calls and literals support typed unpack/spread forms. | Manually spelling every forwarding arity or merging collections one element at a time. | [Functions and calls](functions.md), [Release 0.3](../../release_notes/0_3.md) | -| User-defined decorators | Syntax | 0.3 | None for user-defined decorators; compiler-owned decorators keep their documented imports. | `@logged`
`@route("/users")`
`@trace(level=Level.INFO)` | Decorators are ordinary callable values applied to functions and methods, including decorator factories. | Boilerplate wrapper declarations around every function that needs the same callable transform. | [Language reference](language.md#decorators), [Derives and traits](derives_and_traits.md), [Release 0.3](../../release_notes/0_3.md) | +| User-defined decorators | Syntax | 0.3 | None for user-defined decorators; compiler-owned decorators keep their documented imports. | `@logged`
`@registered("catalog.ref")`
`func.__name__`
`@registered[(str) -> ColumnExpr]("catalog.ref")` | Decorators are ordinary callable values applied to functions and methods, including generic decorator factories that infer or accept the decorated function type and decorator helpers that expose `func.__name__`. | Boilerplate wrapper declarations around every function that needs the same callable transform. | [Language reference](language.md#decorators), [Derives and traits](derives_and_traits.md), [Release 0.3](../../release_notes/0_3.md) | | Generators | Syntax | 0.3 | None. | `def numbers() -> Generator[int]:`
`yield value`
`(x * 2 for x in values)` | `yield`-based functions and generator expressions produce lazy `Generator[T]` values. | Eager list construction when callers only need lazy iteration. | [Generators](generators.md), [Generators how-to](../how-to/generators.md), [Release 0.3](../../release_notes/0_3.md) | | Iterator adapters and terminal consumers | Stdlib | 0.3 | Use iterator values. | `values.iter().map(parse).filter(valid).collect()`
`items.enumerate().take(10)`
`numbers.fold(0, add)` | Iterator pipelines expose lazy adapters and explicit terminal consumers. | Manual loop accumulators for ordinary map/filter/fold pipeline shapes. | [Collection protocols](stdlib_traits/collection_protocols.md), [Release 0.3](../../release_notes/0_3.md) | | `Result[T, E]` combinators | Stdlib | 0.3 | Use `Result[T, E]` values. | `result.map(transform)`
`result.and_then(validate)`
`result.inspect(log_success)` | `Result` values support branch-local transforms, fallible chaining, recovery, and inspection taps. | Nested matches that only rewrap `Ok` / `Err` around one transformed branch. | [std.result](stdlib/result.md), [Fallible and infallible paths](../tutorials/fallible_and_infallible_paths.md), [Release 0.3](../../release_notes/0_3.md) | @@ -464,13 +464,14 @@ Canonical forms: - **Use instead of:** Boilerplate wrapper declarations around every function that needs the same callable transform. - **References:** [Language reference](language.md#decorators), [Derives and traits](derives_and_traits.md), [Release 0.3](../../release_notes/0_3.md) -Decorators are ordinary callable values applied to functions and methods, including decorator factories. +Decorators are ordinary callable values applied to functions and methods, including generic decorator factories that infer or accept the decorated function type and decorator helpers that expose `func.__name__`. Canonical forms: - `@logged` -- `@route("/users")` -- `@trace(level=Level.INFO)` +- `@registered("catalog.ref")` +- `func.__name__` +- `@registered[(str) -> ColumnExpr]("catalog.ref")` ### Generators diff --git a/workspaces/docs-site/docs/language/reference/language.md b/workspaces/docs-site/docs/language/reference/language.md index 9eb7038c2..c9f1da930 100644 --- a/workspaces/docs-site/docs/language/reference/language.md +++ b/workspaces/docs-site/docs/language/reference/language.md @@ -289,7 +289,7 @@ def main() -> None: ## Decorators -User-defined decorators are valid on top-level `def` / `async def` declarations and instance methods. A decorator is an ordinary callable value that receives the decorated function value and returns the binding that should replace it: +User-defined decorators are valid on top-level `def` / `async def` declarations and instance methods. A decorator is an ordinary callable value that receives the decorated function or method callable and returns the callable that should replace it: ```incan def parse(value: int) -> int: @@ -308,6 +308,45 @@ def main() -> None: Stacked decorators apply bottom-up, matching Python's declaration model: the decorator closest to `def` receives the original function value first, and the outer decorators receive each previous result. Decorator factories such as `@logged("name")` are checked by first evaluating the factory expression as a callable-producing expression and then applying the produced decorator to the function value. +!!! tip "Coming from Python?" + Python decorators can replace a function with any object. Incan user-defined function decorators are stricter: the decorator input is the decorated callable, and the result must also be callable. Python's `Callable[[A, B], R]` corresponds to Incan's `(A, B) -> R`; `=>` is only for closure expressions, not callable types. Use `(F) -> F` when a decorator preserves the original callable signature, and spell the source and replacement callable types separately when it intentionally changes the signature, such as `((str) -> R) -> ((str, str) -> R)`. + +Decorator factories can be generic over the decorated function type. This is the usual shape for registry, catalog, routing, telemetry, and validation decorators that record metadata but return the original function unchanged: + +```incan +def registered[F](function_ref: str) -> ((F) -> F): + return (func) => func + +@registered("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +The compiler infers `F` from the decorated function when the factory result is applied. If inference needs help, pass the decorated function type explicitly on the decorator factory call: + +```incan +@registered[(str) -> ColumnExpr]("inql.functions.col") +pub def col(name: str) -> ColumnExpr: + return ColumnExpr(name=name) +``` + +The post-decoration binding keeps the concrete callable signature of the decorated function unless the decorator deliberately returns a different callable shape. Checked API metadata and imports observe that concrete signature, not the generic helper's `F`. + +The callable value passed into a decorator exposes `__name__` as the source callable name. Registry and catalog decorators can use this from concrete decorator helpers and from generic `(F) -> F` helpers, so a decorator can record `func.__name__` without requiring the decorated declaration to repeat its own public name in a string argument. + +```incan +def capture[F](func: F) -> F: + registry_names.append(func.__name__) + return func + +def registered[F]() -> ((F) -> F): + return (func) => capture[F](func) + +@registered() +pub def sample(value: int) -> int: + return value + 1 +``` + Method decorators receive an unbound callable shape with the receiver first. A decorator on `def label(self, value: int) -> str` sees `(&Box, int) -> str`; a decorator on `def bump(mut self, value: int) -> int` sees `(&mut Box, int) -> int`. The wrapper passes the actual receiver borrow through to the decorated callable, so method decorators do not require cloning the receiver. Class, model, trait, enum, newtype, field, alias, and module decorators remain limited to compiler-owned decorators. Compiler-owned decorators such as `@derive`, `@route`, `@rust.extern`, `@rust.allow`, `@staticmethod`, `@classmethod`, and `@requires` keep their existing special behavior. @@ -641,6 +680,35 @@ Class, model, trait, enum, newtype, field, alias, and module decorators remain l | OrElse | `or_else` | | Recover or remap through a Result-returning operation from an Err payload. | RFC 070 | 0.3 | Stable | | Inspect | `inspect` | | Observe an Ok payload by implicit borrow while preserving the original Result. | RFC 070 | 0.3 | Stable | | InspectErr | `inspect_err` | | Observe an Err payload by implicit borrow while preserving the original Result. | RFC 070 | 0.3 | Stable | +| Unwrap | `unwrap` | | Return the Ok payload or panic. | RFC 000 | 0.1 | Stable | +| UnwrapOr | `unwrap_or` | | Return the Ok payload or a default value. | RFC 000 | 0.1 | Stable | + + +### Iterator methods + +| Id | Canonical | Aliases | Description | RFC | Since | Stability | +|---|---|---|---|---|---|---| +| Iter | `iter` | | Create an iterator over an iterable. | RFC 088 | 0.3 | Stable | +| Map | `map` | | Lazily transform iterator items. | RFC 088 | 0.3 | Stable | +| Filter | `filter` | | Lazily keep items that match a predicate. | RFC 088 | 0.3 | Stable | +| Enumerate | `enumerate` | | Yield each item with its zero-based index. | RFC 088 | 0.3 | Stable | +| Zip | `zip` | | Pair items from two iterables. | RFC 088 | 0.3 | Stable | +| Take | `take` | | Yield at most the requested number of items. | RFC 088 | 0.3 | Stable | +| Skip | `skip` | | Discard at most the requested number of items. | RFC 088 | 0.3 | Stable | +| TakeWhile | `take_while` | | Yield items until a predicate first returns false. | RFC 088 | 0.3 | Stable | +| SkipWhile | `skip_while` | | Discard items while a predicate returns true. | RFC 088 | 0.3 | Stable | +| Chain | `chain` | | Yield receiver items followed by another iterable. | RFC 088 | 0.3 | Stable | +| FlatMap | `flat_map` | | Map items to iterables and flatten the result. | RFC 088 | 0.3 | Stable | +| Batch | `batch` | | Yield fixed-size list batches. | RFC 088 | 0.3 | Stable | +| Collect | `collect` | | Consume an iterator into a list. | RFC 088 | 0.3 | Stable | +| Count | `count` | | Consume an iterator and return the item count. | RFC 088 | 0.3 | Stable | +| Reduce | `reduce` | | Consume an iterator with an explicit accumulator. | RFC 088 | 0.3 | Stable | +| Fold | `fold` | | Consume an iterator with an explicit accumulator. | RFC 088 | 0.3 | Stable | +| Any | `any` | | Return whether any item satisfies a predicate. | RFC 088 | 0.3 | Stable | +| All | `all` | | Return whether every item satisfies a predicate. | RFC 088 | 0.3 | Stable | +| Find | `find` | | Return the first item satisfying a predicate. | RFC 088 | 0.3 | Stable | +| ForEach | `for_each` | | Consume an iterator for side effects. | RFC 088 | 0.3 | Stable | +| Sum | `sum` | | Consume an iterator and return the numeric sum. | RFC 088 | 0.3 | Stable | ### FrozenList methods diff --git a/workspaces/docs-site/docs/release_notes/0_3.md b/workspaces/docs-site/docs/release_notes/0_3.md index d32bad5ec..4ad5e64cb 100644 --- a/workspaces/docs-site/docs/release_notes/0_3.md +++ b/workspaces/docs-site/docs/release_notes/0_3.md @@ -1,598 +1,194 @@ # Release 0.3 -Incan 0.3 picks up after the `0.2` line, which made the language surface more explicit around stdlib imports, Rust interop, library manifests, module state, and call-site generics. +Incan 0.3 builds on the `0.2` line by making larger programs feel less improvised: richer source-level language features, a much broader standard library, a stronger test runner, better Rust interop, and fewer generated-Rust ownership surprises. -`0.3` now includes a larger numeric surface, a new control-flow surface, richer enum behavior, Rust trait adoption from Incan-owned wrappers, graph, collections, datetime, logging, encoding, hashing, compression, regex, and dynamic JSON stdlib surfaces, iterator adapter chains, Result combinators, and tighter tooling contracts. RFC 009 makes numeric annotations precise enough for Rust interop, wire formats, data schemas, and fixed-scale decimal values; RFC 016 adds `loop:` and `break ` so loops can produce values directly; RFC 030 adds `std.collections` for specialized collection semantics; RFC 101 extends that collections surface with `OrdinalMap` for deterministic immutable key-to-ordinal lookup; RFC 047 adds `std.graph` for explicit in-memory dependency and plan graphs; RFC 064 adds `std.encoding` for strict-by-default binary-text transforms; RFC 065 adds `std.hash` for stable byte, file, reader, cryptographic, compatibility, and non-cryptographic hashing workflows; RFC 061 adds `std.compression` for byte, stream, and explicit autodetected compression workflows; RFC 059 adds safe-default regular expressions with explicit match/capture/replacement contracts; RFC 051 adds `std.json.JsonValue` for dynamic parse-inspect-transform workflows and `std.math` numeric-like string classification helpers; RFC 050 lets enums declare methods and adopt traits; RFC 043 starts Rust trait implementation authoring from Incan source with `with Trait`, method-level `for Trait`, and associated type declarations on newtypes and rusttypes; RFC 088 standardizes lazy iterator adapters and terminal consumers; RFC 070 adds Rust-shaped `Result[T, E]` composition with `map`, `map_err`, `and_then`, `or_else`, `inspect`, and `inspect_err`; RFC 053 tightens the formatter contract so output is less dependent on local heuristics and more predictable across CLI, editor, and library entry points; RFC 058 adds Rust-backed runtime timing plus source-defined civil temporal values, fixed UTC offsets, Python-shaped parsing/formatting, and interval arithmetic; and RFC 072 adds source-defined structured logging. +If `0.2` was mostly about explicit namespaces, library manifests, and Rust boundary cleanup, `0.3` is about using that structure for real application code. The release adds typed numerics, expression-oriented control flow, enum behavior, protocol hooks, Rust trait adoption, graph/collection/JSON/regex/datetime/logging/encoding/hash/compression stdlib modules, lazy iterator pipelines, `Result` combinators, first-class testing workflows, and tighter lockfile/formatter/tooling contracts. -If you are looking for the shipped `0.2` story, start with [Release 0.2](0_2.md). - -For numeric guidance, start with [Choosing numeric types](../language/how-to/choosing_numeric_types.md) and [Numeric semantics](../language/reference/numeric_semantics.md). For the current control-flow guidance, start with [Control Flow](../language/explanation/control_flow.md). For the current source-layout contract, start with the [Incan Code Style Guide](../language/reference/code_style.md). Use [Formatting with `incan fmt`](../tooling/how-to/formatting.md) for the tool behavior. [RFC 009], [RFC 016], and [RFC 053] record the design snapshots behind those behaviors. +If you are looking for the previous release story, start with [Release 0.2](0_2.md). For current user docs, start with [Control Flow](../language/explanation/control_flow.md), [Choosing numeric types](../language/how-to/choosing_numeric_types.md), [Testing in Incan](../language/how-to/testing_stdlib.md), the [standard library reference](../language/reference/stdlib/index.md), and [Rust interop](../language/how-to/rust_interop.md). ## What 0.3 is about -The `0.2` line made Incan's module, stdlib, and Rust interop boundaries much clearer. `0.3` continues from that baseline with a stronger emphasis on predictable generated output, contributor ergonomics, and small but meaningful control-flow ergonomics that remove repetitive boilerplate without weakening the language's explicit pattern model. - -The release emphasizes a few concrete directions: - -- expression-oriented control flow should stay explicit, so infinite loops that return values use `loop:` and `break ` rather than hidden accumulator patterns -- numeric annotations should be ergonomic for ordinary code while still exact enough for Rust APIs, binary formats, database schemas, and analytics data -- formatter output should be governed by explicit contracts, not scattered newline decisions -- common `Option` / `Result` destructuring should have a concise control-flow form when the non-match case is intentionally a no-op -- enums that cross string or integer boundaries should keep enum type safety while exposing one canonical raw representation -- enum-owned behavior should live on the enum itself, and enums should be able to adopt the same trait protocols as models and classes -- Rust ecosystem trait contracts should be authored through Incan's `with` adoption model where possible, with Rust `impl` blocks treated as generated backend output -- operator overloading should present traits as nominal capability contracts while keeping dunder methods as the explicit implementation hooks -- in-memory graph-shaped data should have one small stdlib vocabulary for node ids, edge ids, directed graphs, DAGs, multigraphs, adjacency, traversal, and cycle-aware ordering -- time-shaped data should have one stdlib vocabulary for Rust-backed runtime timing, civil dates and times, fixed UTC offsets, and calendar-aware intervals -- binary-text encoding should have explicit stdlib modules for strict and lenient value plus finite source/sink transforms, with variant choices visible in API names -- specialized collection semantics should have explicit stdlib types instead of forcing every queue, multiset, ordered map, sorted set, layered map, or priority queue through bare builtin containers -- immutable key-to-ordinal lookup should have an explicit deterministic contract instead of being modeled as an ad hoc `dict[K, int]` when serialized bytes, stable scalar key encodings, exact safe lookup, and compact ordinal storage matter -- byte, file, and reader hashing should have explicit algorithm namespaces, with cryptographic and compatibility digests separated from fast non-cryptographic integer helpers -- compression should be codec-explicit by default, with stream helpers and explicit autodetection rather than hidden format guessing -- user-facing tooling behavior should match the docs closely enough that CI and editor integrations can rely on it -- testing should feel like a first-class workflow, with inline unit tests, fixtures, parametrization, selection, scheduling, and machine-readable reports owned by Incan rather than delegated to ad hoc scripts -- iterator pipelines should be lazy by default, with terminal consumers such as `.collect()`, `.count()`, `.any()`, `.all()`, `.find()`, and `.fold()` making realization or summarization explicit -- `Result` pipelines should support branch-local transforms, fallible chaining, recovery, and inspection taps without requiring repetitive nested `match` scaffolding -- ownership and generated-runtime ergonomics should improve structurally, not through one-off `.clone()` or `.as_ref()` patches -- standard-library filesystem workflows should distinguish ordinary path/file operations from temporary resource acquisition and cleanup -- regular-expression workflows should use one safe stdlib vocabulary for compiled patterns, captures, splitting, and replacement instead of pushing ordinary text processing through Rust interop -- application and library logging should use one structured stdlib vocabulary instead of pushing ordinary Incan code through Rust logging interop -- dynamic JSON payloads should have one explicit stdlib value type instead of ad hoc dictionaries or schema-shaped models for data whose shape is intentionally open - -## Migrating from 0.2 - -There are no required source migrations for ordinary `int` and `float` code. Those spellings remain valid and keep their `i64` / `f64` representations. - -Numeric annotations can now be more specific when the representation matters. Code that used a project-local bare type name `decimal` or `numeric` should rename that type or use the new parameterized forms such as `decimal[12, 2]`; those bare names are now reserved for the numeric type family. Data-shaped aliases such as `bigint`, `hugeint`, `integer`, `smallint`, `real`, and `double` canonicalize to exact Incan types rather than introducing new nominal types. - -`loop:` and `break ` are additive control-flow features; existing `while True:` code remains valid. - -Projects that gate on `incan fmt --check` should expect one-time vertical-spacing diffs when adopting a formatter that implements RFC 053. Those diffs are intentional: top-level `def` / `model` / `type`-like declarations get exactly two blank lines around them, following body-bearing members inside type bodies get exactly one blank line, and other same-scope transitions stay in the zero-or-one bucket. - -`if let` and `while let` are additive. Existing `match` code keeps working unchanged; the new forms are available when a single successful pattern matters and the non-match path should do nothing. - -## Major additions - -### RFC 101 `std.collections.OrdinalMap` - -Incan now has `std.collections.OrdinalMap[K]` for immutable deterministic lookup from a stable key domain to integer ordinals. - -```incan -from std.collections import OrdinalMap - -columns = OrdinalMap.from_keys(["order_id", "customer_id", "status", "amount"])? - -assert columns.require("status")? == 2 -assert columns.get("missing") == None -``` - -`OrdinalMap` is for schemas, catalogs, generated metadata, dictionary-encoded scalar domains, and cached lookup tables whose bytes must be reproducible. It is not a mutable `dict` replacement. Keys implement `OrdinalKey`, which supplies deterministic canonical bytes and a stable encoding identifier for serialization. The supported scalar surface includes `str`, `bytes`, `bool`, integers, fixed-precision decimals, UUID values, date/time values, stable value enums, and user-defined adopters. Floating-point keys remain outside the contract for now. - -Safe lookup is exact through `get`, `require`, membership, indexing, and batch lookup. Unchecked lookup is explicit and non-default for callers that have already proven key presence. `from_keys` rejects duplicate keys, and `from_pairs` rejects duplicate keys, negative ordinals, and duplicate ordinals. - -Serialization is deterministic and uses the `INCAN_ORDMAP` container. The payload records format metadata, the key encoding identifier, exact-verification data, and compact ordinal cells selected from the maximum ordinal (`u8`, `u16`, `u32`, or `u64`), while public lookup returns ordinary `int`. - -See also: [`std.collections`](../language/reference/stdlib/collections.md), [Choosing collection types](../language/how-to/choosing_collections.md), [Why `OrdinalMap` exists](../language/explanation/ordinal_map.md), [RFC 101](../RFCs/closed/implemented/101_std_collections_ordinal_map.md). - -### RFC 051 `std.json.JsonValue` - -Incan now has `std.json.JsonValue` for dynamic JSON payloads whose complete shape is not known at compile time. `JsonValue` complements typed `@derive(json)` models: keep stable fields typed and use `JsonValue` for open, exploratory, or mixed-shape fields. - -```incan -from std.serde import json -from std.json import JsonValue - -@derive(json) -model Envelope: - status: int - data: JsonValue -``` - -The surface includes parsing, compact and pretty serialization, constructors for every JSON kind, `JsonKind` inspection, typed extraction helpers, object and array mutation helpers, JSON Pointer traversal, deterministic display/debug behavior, and JSON-specific errors. Direct indexing is checked and optional: `value["key"]` and `value[0]` return `Option[JsonValue]`, preserving the distinction between a missing key and a present JSON null. - -JSON number parsing follows the same JSON-compatible lexical contract exposed by shared `std.math.is_int_like(value: str)` and `std.math.is_float_like(value: str)` helpers. Integer-like JSON numbers map to Incan `int`; fractional or exponent forms map to Incan `float`. - -See also: [`std.json`](../language/reference/stdlib/json.md), [`std.math`](../language/reference/stdlib/math.md), [Derives: Serialization](../language/reference/derives/serialization.md), [RFC 051]. - -### RFC 072 `std.logging` - -Incan now has a `std.logging` module for ordinary structured logging. Code can emit through the ambient `log` surface for the current module's default logger, acquire explicit named loggers with `get_logger(...)`, and attach structured primitive fields or `std.telemetry.core.TelemetryValue` fields at each call site. - -```incan -from std.logging import Level, basic_config - -def main() -> None: - basic_config(level=Level.INFO, target="stdout") - log.info("started", fields={"component": "worker"}) -``` - -Logger values, validated `LoggerName` and `OutputTarget` values, source-level configuration, level filtering, bound context, human rendering, and JSON rendering are implemented in Incan stdlib source. Log records use the `std.telemetry.core` data model and OpenTelemetry log data model aliases for JSON output; `Level.WARN` and `Level.FATAL` are canonical, with `WARNING` and `CRITICAL` as enum variant aliases. The module uses `std.datetime` for timestamps and ordinary `rust::std::io` imports for stdout/stderr delivery without adding a logging-specific Rust backing module. Project defaults, environment overrides, CLI logging flags, and colorized terminal policy remain future host-boundary work. - -See also: [`std.logging`](../language/reference/stdlib/logging.md), [Logging](../language/how-to/logging.md), [RFC 072]. - -### RFC 059 `std.regex` - -Incan now has a `std.regex` module for compiled, reusable regular expressions over `str`. - -```incan -from std.regex import Regex, RegexError - -def main() -> Result[None, RegexError]: - release = Regex("^v(?P\\d+)\\.(?P\\d+)$")? - caps = release.full_match("v0.3") - - match caps: - Some(version) => - println(version.group("major").unwrap_or("")) - None => - println("not a release tag") - - return Ok(None) -``` - -The stdlib surface is intentionally a safe-default engine contract, aligned with the predictable Rust-regex/RE2-style family rather than a fully backtracking Python/PCRE-style engine. It supports ordinary literals, character classes, quantifiers, alternation, grouping, anchors, named captures, indexed captures, Unicode-aware matching, inline flags, and constructor flags such as `ignore_case`, `multiline`, `dotall`, and `verbose`. Lookaround and pattern backreferences are outside the `std.regex` contract. - -`Match` exposes matched text and spans. `Captures` exposes group `0` for the full match, indexed and named group lookup, capture spans, `groups()`, and `groupdict()`. Unmatched optional capture groups remain explicit `None` values instead of silently becoming empty strings. Split APIs return iterators, and replacement supports first/all/limited replacement with replacement-string interpolation (`$1`, `${name}`) or callable replacements that receive `Captures`. - -See also: [`std.regex`](../language/reference/stdlib/regex.md), [Regular expressions](../language/how-to/regular_expressions.md), [Strings and bytes](../language/reference/strings.md), [Callable objects](../language/reference/stdlib_traits/callable.md), [RFC 059](../RFCs/closed/implemented/059_std_regex.md). - -### RFC 009 numeric type system - -Incan now has a registry-backed numeric type system instead of only the broad `int` and `float` spellings. Use `int` and `float` for ordinary Incan-owned logic, then opt into exact widths when the number crosses a contract boundary. - -```incan -attempts: int = 3 -timeout_seconds: float = 2.5 - -packet_version: u8 = 1 -message_count: u32 = 4096 -rust_code: i32 = 200 -``` - -The new canonical integer surface covers `i8`, `i16`, `i32`, `i64`, `i128`, `u8`, `u16`, `u32`, `u64`, `u128`, `isize`, and `usize`. Binary floats now include `f32` and `f64`. `int` remains the ergonomic signed integer spelling and canonicalizes to `i64`; `float` remains the ergonomic binary float spelling and canonicalizes to `f64`. - -Data and analytics vocabulary is also recognized where it maps cleanly to an exact type: - -```incan -model WarehouseRow: - id: bigint - fingerprint: hugeint - category_id: integer - priority: smallint - score: double -``` - -Aliases do not create separate runtime types. `integer` is `i32`, `bigint` is `i64`, `hugeint` is `i128`, `real` is `f32`, and `double` is `f64`. - -Fixed-scale decimal annotations are now accepted with explicit precision and scale: - -```incan -unit_price: decimal[12, 2] = 19.99d -tax_rate: numeric[6, 4] = 0.0825d -``` - -The compiler validates decimal precision, scale, and literal fit. Decimal values lower through the toolchain-owned `Decimal128` representation, so they are useful for typed boundaries, literal validation, formatting, generated Rust, and display. General decimal arithmetic is not yet part of the language contract. - -Numeric conversion is intentionally explicit when values may be lost. Lossless widening is accepted, including at Rust interop boundaries, but narrowing and sign-changing conversions require a policy: - -```incan -small: i8 = 120 -wide: int = small.resize() - -incoming: int = 240 -maybe_small: Option[i8] = incoming.try_resize() -wrapped: i8 = incoming.wrapping_resize() -capped: i8 = incoming.saturating_resize() -``` - -The practical rule is simple: write ordinary business logic with `int` and `float`, match exact widths at external boundaries, use schema-shaped aliases when they make data models read like their source schema, and choose a resize policy before narrowing. - -See also: [Numeric semantics](../language/reference/numeric_semantics.md), [Choosing numeric types](../language/how-to/choosing_numeric_types.md), [Why numeric types work this way](../language/explanation/numeric_types.md), [Rust interop](../language/how-to/rust_interop.md), [RFC 009]. - -### RFC 016 loop expressions and `break ` - -Incan now has an explicit infinite-loop construct: - -- `loop:` for intentional infinite loops in statement position -- `break ` to complete a `loop:` expression with a result -- ordinary `break` and `continue` continuing to work for `for`, `while`, and statement-form `loop:` - -This makes "search until found", retry loops, and similar control-flow patterns expression-oriented without forcing a mutable accumulator outside the loop. - -See also: [Control Flow](../language/explanation/control_flow.md), [Book chapter 4](../language/tutorials/book/04_control_flow.md), [RFC 016]. - -### RFC 053 formatter vertical-spacing contract - -`incan fmt` now follows RFC 053's three-bucket vertical-spacing model: - -- **Exactly two blank lines** around top-level `def`, `class`, `model`, `trait`, `enum`, `type`, `newtype`, and `rusttype` declarations -- **Exactly one blank line** before a following body-bearing member inside a type body -- **At most one blank line** everywhere else, including import runs, adjacent constants/statics, ordinary statement blocks, and transitions involving module docstrings when no top-level spaced declaration is involved - -The formatter also normalizes docstring payload indentation while collapsing actual docstring blank-line runs to one blank line, keeps abstract trait methods tight until a following default/body-bearing method, treats stand-alone comments as leading or trailing bundles even when their target statement wraps, preserves a single authored blank line between statement groups after nested suites, keeps short single-statement `match` arms inline, normalizes blank lines after suite headers and match-arm arrows, strips trailing blank lines at EOF, and allows two consecutive blank lines only at root level. - -Long call-like expressions and signatures now participate in formatter wrapping: overflowing constructor calls, ordinary calls, function signatures, and method signatures are rewritten across multiple lines and respect the existing trailing-comma setting. - -The same spacing contract applies through the CLI and the library formatter API. `FormatConfig` still controls ordinary formatting options such as indentation and line length, but vertical-spacing buckets and comment placement are not configurable. - -See also: [Incan Code Style Guide](../language/reference/code_style.md), [Formatting with `incan fmt`](../tooling/how-to/formatting.md), [RFC 053]. - -### RFC 049 `if let` and `while let` control flow - -Incan now supports `if let PATTERN = VALUE:` and `while let PATTERN = VALUE:` in statement position. - -Use `if let` when exactly one successful pattern matters and the non-match path should do nothing. Use `while let` when a loop should keep iterating only while one pattern keeps matching. Both forms reuse the same pattern semantics as `match`, keep bindings scoped to the successful branch or loop body, and leave full `match` as the right tool when multiple arms or explicit non-match behavior matter. - -In v1, `if let` remains intentionally single-arm only and rejects `else` / `elif`. When the non-match path is semantically important, keep using `match`. - -### RFC 029 union types and narrowing - -Incan now accepts anonymous closed union annotations with both canonical `Union[A, B, ...]` and `A | B` syntax. Concrete member values can flow into union-typed returns and bindings, source unions can flow into wider target unions, and unions containing `None` canonicalize through `Option[...]`. - -Union values must be narrowed before using member-specific methods. The compiler now supports `isinstance(value, T)` narrowing for true branches, else branches, wider unions, chained `elif` branches, and `Option[Union[...]]` values; `is None` / `is not None` narrowing for `Option[...]`-canonicalized unions; and `match` type patterns such as `int(n)` and `str(s)`, with exhaustiveness checking for ordinary unions. - -### RFC 032 value enums - -Incan now supports value enums with `str` and `int` backing values: - -```incan -enum Environment(str): - Development = "development" - Production = "production" - -enum HttpStatus(int): - Ok = 200 - NotFound = 404 -``` - -Value enum variants remain enum values. They are not subtypes of the backing primitive and do not compare equal to raw primitive values. The generated `value()` helper returns the canonical raw value, while `from_value(...)` returns `Option[Enum]` for explicit handling of unknown external values. Generated display, string parsing, and serde hooks use the raw representation for value enums. - -### RFC 050 enum methods and trait adoption - -Enums can now declare methods and associated functions inside the enum body, after their variants. Use this when behavior belongs to the closed set itself, such as `Direction.opposite()` or `BuildState.describe()`, instead of pushing enum-owned logic into detached helper functions. +The main direction is not "more syntax for its own sake." `0.3` moves common project patterns into documented language and stdlib surfaces so users can write less Rust-shaped scaffolding and contributors can keep compiler behavior tied to explicit metadata. -Enums can also adopt traits with `with TraitName`, using the same trait adoption surface as models and classes. This makes enum-backed protocols reusable without special-case compiler support while keeping existing enum semantics additive and variant sets closed. +- **Language**: Numeric widths, fixed-scale decimal annotations, `loop:` expressions, `if let` / `while let`, union narrowing, value enums, enum methods, computed properties, decorators, aliases, partial callables, protocol hooks, variadics, generators, and pattern alternation make the Python-shaped surface more expressive while staying statically checked. +- **Stdlib**: Collections, `OrdinalMap`, graphs, JSON values, regex, datetime, logging, encoding, hashing, compression, filesystem, I/O, temporary files, UUIDs, iterator adapters, and `Result` helpers move ordinary application needs out of ad hoc Rust interop. +- **Interop**: Rust crate imports, `rusttype`, trait adoption, associated types, derived Rust metadata, metadata-backed call boundaries, and generated Rust retention now cooperate better with real Rust crates and protobuf-style APIs. +- **Tooling**: `incan test`, `incan fmt`, `incan lock`, lifecycle commands, doctor diagnostics, checked API metadata, LSP metadata, and generated Rust audits are more deterministic and CI-friendly. +- **Architecture**: More behavior is registry- and metadata-driven, and generated Rust relies less on scattered special cases. -### RFC 043 Rust trait adoption from Incan - -Incan can now start expressing Rust trait implementations from Incan source on newtype and rusttype declarations. Authors use the existing `with TraitName` adoption clause instead of writing Rust-shaped `impl Trait for Type` source syntax. - -```incan -from rust::std::fmt import Debug, Display, Formatter, FmtError - -type UserId = rusttype i64 with Display, Debug: - def fmt(self, f: Formatter) for Display -> Result[None, FmtError]: - return f.write_str(f"user_{self.0}") - - def fmt(self, f: Formatter) for Debug -> Result[None, FmtError]: - return f.write_str(f"UserId({self.0})") -``` - -The method-level `for TraitName` target is only needed when more than one adopted trait could claim the same method name. Associated type declarations also use Incan syntax, for example `type Output for Add[int] = UserId`. - -The compiler also validates imported Rust trait metadata for associated type requirements, rejects statically knowable Rust coherence violations, forwards supported `@rust.derive(...)` attributes to generated Rust items, accepts metadata-proven body-less `rusttype` forwarding without emitting invalid alias impls, and explicitly gates RFC 039 `Awaitable[T]` to Rust `Future` bridging until safe pin-projection and output-mapping metadata exist. - -### RFC 028 trait-based operator overloading - -`std.traits.ops` now exposes the RFC 028 operator protocol vocabulary for custom types. The basic arithmetic traits are joined by floor division, power, shifts, bitwise operators, matrix multiplication, pipe operators, unary inversion, indexing hooks, and explicit in-place compound-assignment traits for the supported `+=`, `-=`, `*=`, `/=`, `//=`, `%=`, `@=`, `&=`, `|=`, `^=`, `<<=`, and `>>=` syntax. - -Operator traits are nominal capability contracts for generic code. Dunder methods such as `__add__`, `__floordiv__`, `__rshift__`, `__matmul__`, and `__getitem__` are the implementation hooks that satisfy those contracts. Compound assignment first checks for the explicit in-place hook such as `__iadd__`; if none exists, it falls back to ordinary binary operator assignment. - -### RFC 068 protocol hooks for core syntax - -Core syntax now resolves through static protocol hooks for user-defined types. Custom types can participate in truthiness, `len(...)`, membership, iteration, indexing, indexed assignment, and callable-object invocation by defining compatible hooks such as `__bool__`, `__len__`, `__contains__`, `__iter__`, `__next__`, `__getitem__`, `__setitem__`, and `__call__`. - -The hook surface remains statically checked. Dunder methods are implementation hooks, while traits such as `Bool`, `Len`, `Contains`, `Iterable`, `Iterator`, `Index`, `IndexMut`, and fixed-arity callable traits are the nominal capability vocabulary for explicit adoption, bounds, docs, and diagnostics. `Option` and `Result` remain intentionally non-truthy; use explicit pattern checks for optionality and fallibility. - -### RFC 058 `std.datetime` - -`std.datetime` now provides temporal value types for runtime timing, civil dates and times, fixed UTC offsets, and interval arithmetic. The module includes `Duration`, `Instant`, `SystemTime`, `Date`, `Time`, `DateTime`, `FixedOffset`, `DateTimeOffset`, `TimeDelta`, `YearMonthInterval`, and `DateTimeInterval`. UTC host-clock civil factories are available as `Date.utc_today()` and `DateTime.utc_now()`; timezone-aware local `today` / `now` semantics remain package-level functionality. - -The runtime timing layer uses Rust `std::time` through ordinary Incan Rust interop for `Duration`, `Instant`, and `SystemTime`. The civil calendar layer remains source-defined Incan, with ISO-style parsing/formatting, Python-shaped `strftime` / `strptime`, nanosecond `%f`, fixed-offset `%z` / `%:z`, comparison, date arithmetic, and interval normalization. Named timezone rule lookup is intentionally left to separately versioned packages. - -See also: [Dates and times](../language/tutorials/dates_and_times.md), [Dates and times how-to](../language/how-to/dates_and_times.md), [std.datetime reference](../language/reference/stdlib/datetime.md), [RFC 058]. - -### RFC 088 iterator adapter surface - -Iterator values now expose the RFC 088 adapter surface for lazy pipelines: `.map()`, `.filter()`, `.flat_map()`, `.take()`, `.skip()`, `.chain()`, `.enumerate()`, `.zip()`, `.take_while()`, `.skip_while()`, and `.batch()`. - -Terminal consumers make realization or summarization explicit with `.collect()`, `.count()`, `.reduce()`, `.fold()`, `.any()`, `.all()`, `.find()`, `.for_each()`, and `.sum()`. Terminal methods consume the iterator, so code that needs to keep the iterator for another pass should call `.clone()` before the terminal operation. `.sum()` supports `int`, `float`, and newtypes over summable underlying types; checked newtypes go through their normal construction validation. For now, `.collect()` returns `list[T]`; it does not accept a target collection type. - -### RFC 070 Result combinators - -`Result[T, E]` now exposes the standard Rust-shaped composition surface: `.map()`, `.map_err()`, `.and_then()`, `.or_else()`, `.inspect()`, and `.inspect_err()`. - -Use `.map()` to transform an `Ok(T)` value, `.map_err()` to transform an `Err(E)` value, `.and_then()` to chain a `Result`-returning success continuation, and `.or_else()` to recover from or remap a failure with a `Result`-returning error continuation. Use `.inspect()` and `.inspect_err()` for logging, metrics, and debugging taps that observe one branch and return the original `Result` unchanged; the compiler passes the observed payload through an implicit borrow so the original branch value remains available to the pipeline. - -Callable arguments are documented with `Callable[...]` vocabulary: for example, `.map()` accepts `Callable[T, U]`, `.map_err()` accepts `Callable[E, F]`, `.and_then()` accepts `Callable[T, Result[U, E]]`, `.or_else()` accepts `Callable[E, Result[T, F]]`, `.inspect()` accepts `Callable[T, None]`, and `.inspect_err()` accepts `Callable[E, None]`. Incan intentionally keeps the Rust method names and does not add Python-style aliases. - -See also: [Fallible and infallible paths](../language/tutorials/fallible_and_infallible_paths.md), [Error handling](../language/explanation/error_handling.md), [Callable objects](../language/reference/stdlib_traits/callable.md), [RFC 070](../RFCs/closed/implemented/070_result_combinators_for_result_types.md). - -### Duckborrowing and ownership-aware codegen - -The backend now routes more generated-Rust ownership decisions through a centralized "duckborrowing" planner. This strengthens the compiler's ability to choose moves, borrows, mutable borrows, owned string materialization, `.into()`, and necessary `.clone()` calls at typed use sites instead of relying on scattered emitter-local fixes. - -Practically, this reduces the need for users and library authors to add ownership-shaping workarounds such as `.clone()`, `.as_ref()`, `str(...)`, or `.into()` in ordinary Incan code just to satisfy generated Rust. The planner now covers more call arguments, collection and tuple literals, assignments, returns, match scrutinees, string lookup probes, tuple unpacking, and Rust interop boundaries. - -### RFC 057 targeted generated-Rust lint suppression - -Incan now supports `@rust.allow(...)` for narrow suppression of specific rustc or Clippy lints on the generated Rust item for one declaration. This is Rust-emission metadata for unavoidable generated-Rust warnings, not arbitrary Rust attribute injection and not project-wide lint configuration. - -The decorator is item-only and covers functions, methods, models, classes, enums, and newtypes. Module-level `rust.allow(...)` directives are not supported. The compiler also rejects obvious broad lint groups including `warnings`, `unused`, `clippy::all`, `clippy::pedantic`, `clippy::nursery`, `clippy::restriction`, and `clippy::cargo`. - -### RFC 010 temporary files and directories - -`std.tempfile` now owns temporary resource creation while `std.fs` owns ordinary path and file operations. Use `NamedTemporaryFile.try_new()` for a named temporary file and `TemporaryDirectory.try_new()` for a temporary directory tree. Use `try_new_with(prefix, suffix, dir)` when the caller needs configured naming or a specific parent directory. Both return `Result[..., IoError]` because they reserve host filesystem entries. - -Temporary wrappers delete their live paths when dropped. Call `path()` to work with the location through `std.fs.Path`, and call `persist()` when the output should survive the wrapper leaving scope. `SpooledTemporaryFile(max_size=...)` starts in memory, rolls over to a named temporary file when it grows beyond `max_size` or `rollover()` is called, and exposes `path()` / `persist()` after rollover. Pathless `TemporaryFile` remains deferred until its cross-platform file-handle contract is settled. - -### Deterministic `incan.lock` files - -`incan.lock` no longer records the wall-clock time when the file was generated. The lock file now contains only reproducibility-relevant inputs such as the Incan lock format version, compiler version, dependency fingerprint, Cargo feature selection, and embedded `Cargo.lock` payload. Re-running `incan lock` against unchanged inputs should leave the file byte-for-byte unchanged, reducing noisy VCS churn in projects that commit lock files. - -Older lock files that still contain the previous `generated = "..."` metadata continue to load, but newly written lock files omit it. - -Default `incan build` and `incan test` also avoid rewriting an existing stale `incan.lock` during routine verification. When the fingerprint differs outside `--locked` / `--frozen`, the command warns and reuses the embedded `Cargo.lock` payload; run `incan lock` when you intentionally want to refresh the committed lock file. - -### RFC 018 testing language primitives - -The language `assert` statement is now an always-on language primitive. Use `assert expr[, msg]` directly for ordinary checks; import `std.testing` when you need helper functions such as `assert_eq`, `assert_is_some`, `fail`, fixtures, parametrization, or marker decorators. - -Testing decorators remain `std.testing` APIs rather than magic global names. `@skip`, `@xfail`, `@slow`, `@fixture`, and `@parametrize` must resolve through `std.testing`, and runner/discovery behavior remains part of RFC 019 rather than RFC 018. - -`assert call() raises ErrorType[, msg]` and compiler-recognized `std.testing.assert_raises[E](block, msg?)` calls now share runtime panic-payload matching. Error payloads match either the exact kind name, such as `ValueError`, or the canonical `Kind: message` prefix. - -### RFC 019 first-class test runner - -`incan test` now has a full runner contract instead of a thin compile-and-run path. Tests can live in conventional `tests/test_*.incn` / `tests/*_test.incn` files or inline `module tests:` blocks inside production source files. Inline tests can exercise same-file private helpers, and production `incan build` / `incan run` output still strips test-only declarations and imports. - -Discovery now supports both `def test_*()` and explicit `@test`, and every collected case has a stable id. Those ids are used consistently by `--list`, `-k`, parametrized test names, JSON Lines output, JUnit XML, and duration reporting. That makes CI logs, reruns, and editor integrations much less dependent on incidental generated-Rust names. - -The runner also picks up the testing ergonomics people expect from a modern test framework: - -- `@fixture` dependency injection, including function, module, and session scopes -- `yield` fixture teardown that can reference setup locals and fixture parameters -- `tests/**/conftest.incn` inheritance for conventional test suites -- built-in `tmp_path`, `tmp_workdir`, and `env` fixtures -- `@parametrize(...)` with stable ids, cartesian products, and `param_case(...)` for per-case ids or marks -- marker selection with `-m`, strict marker registries via `TEST_MARKERS`, and default marks via `TEST_MARKS` -- `@skip`, `@xfail`, `@slow`, `@mark`, `@timeout`, `@resource`, and `@serial` -- collection-time `@skipif` / `@xfailif` probes using `platform()` and `feature("name")` - -Parallel execution is now runner-level and resource-aware. `--jobs N` runs generated worker batches concurrently while each batch still executes through single-threaded libtest. `@resource("name")` prevents overlapping batches that share a resource key, and `@serial` forces exclusive execution. Session fixtures are cached once per worker batch, so `--jobs 1` can reuse a session fixture across compatible collected files, while higher job counts keep one session instance per worker. - -Reporting is also CI-ready. `--format json` emits JSON Lines records with `schema_version: "incan.test.v1"`, `--junit ` writes JUnit XML, `--durations N` reports slow tests, `--shuffle --seed N` gives reproducible randomized order, `--run-xfail` treats expected failures as ordinary tests, and `--nocapture` opts into printing child output for passing tests. Timeout-killed workers can still bypass teardown, so timeout teardown remains best-effort. - -See also: [Testing in Incan](../language/how-to/testing_stdlib.md), [Tooling: Testing](../tooling/how-to/testing.md), [std.testing reference](../language/reference/stdlib/testing.md), [RFC 018], [RFC 019]. - -### RFC 004 async fixtures - -`@fixture` now works on `async def` fixture functions. Async fixtures use the same decorator as synchronous fixtures, use `yield` exactly once, await setup before dependents run, and await teardown after `yield` before the runner continues through reverse dependency teardown. - -Mixed sync and async fixture graphs compose under function, module, and session scopes. Parametrized tests still expand before fixture resolution, so function-scoped async fixtures run per expanded case while module and session fixtures reuse values according to their existing scope boundaries. - -Timeout behavior stays runner-level. `incan test --timeout` and `@timeout(...)` from `std.testing` apply to generated test batches; there is no per-fixture timeout configuration. The runner awaits async fixture teardown after ordinary failures and panics while the worker remains alive, but timeout-enforced worker termination can still bypass remaining cleanup. - -See also: [Testing in Incan](../language/how-to/testing_stdlib.md), [Tooling: Testing](../tooling/how-to/testing.md), [std.testing reference](../language/reference/stdlib/testing.md), [RFC 004](../RFCs/closed/implemented/004_async_fixtures.md). - -### RFC 047 `std.graph` - -`std.graph` now provides a small graph standard-library surface for in-memory dependency, plan, pipeline, and workflow graphs. `DiGraph[T]`, `Dag[T]`, and `MultiDiGraph[T]` are constructed directly with `DiGraph[T]()`, `Dag[T]()`, and `MultiDiGraph[T]()`. `DiGraph[T]` stores typed node payloads behind stable `NodeId` values, `Dag[T]` keeps acyclicity as a data invariant, and `MultiDiGraph[T]` supports parallel directed edges with stable `EdgeId` values. The API exposes adjacency queries, roots, sinks, node and edge removal, breadth-first traversal, depth-first preorder traversal, and cycle-aware topological ordering. - -Graphs are ordinary values rather than ambient singletons. Store them on models, pass them to functions, and keep separate graph instances for separate requests, tests, or pipeline plans. - -The v1 surface is intentionally not a graph database, persistence layer, query language, or distributed graph engine. Future graph expansion remains stdlib design work rather than ad hoc growth. - -See also: [std.graph reference](../language/reference/stdlib/graph.md), [Working with graphs](../language/how-to/working_with_graphs.md), [Why `std.graph` exists](../language/explanation/graph_model.md), [RFC 047]. - -### RFC 030 `std.collections` - -`std.collections` now provides the standard-library namespace for specialized container types that are semantically distinct from builtin `list`, `dict`, and `set`. The module covers `Deque[T]`, `Counter[T]`, `DefaultDict[K, V]`, `OrderedDict[K, V]`, `OrderedSet[T]`, `SortedDict[K, V]`, `SortedSet[T]`, `ChainMap[K, V]`, and `PriorityQueue[T]`. - -These are ordinary Incan stdlib types. They import through `from std.collections import ...`, resolve through the standard stdlib registry and source loader, and do not use Rust-backed stdlib dispatch. - -Use builtin collections for ordinary values. Use `std.collections` when the collection behavior is the point: double-ended queue operations, counted membership, missing-key defaulting, insertion-order stability, sorted traversal, layered configuration, or priority scheduling. - -See also: [std.collections reference](../language/reference/stdlib/collections.md), [Choosing collection types](../language/how-to/choosing_collections.md), [RFC 030]. - -### RFC 064 `std.encoding` - -`std.encoding` now provides the standard-library namespace for binary-text representation transforms. The module covers explicit `hex`, `base32`, `base64`, `base85`, `base58`, and `bech32` families with strict decoding by default, separately named lenient decoders where interoperability needs them, and canonical `encode` / `decode` helpers that work with in-memory values, `std.io.BytesIO`, and finite `std.fs.Path` sources or sinks. - -These are ordinary Incan stdlib APIs. The public surface is source-owned under `std.encoding`, and examples compose with byte/string values and stream types instead of exposing Rust-backed public shells. - -See also: [std.encoding reference](../language/reference/stdlib/encoding.md), [Binary-text encoding](../language/how-to/binary_text_encoding.md), [RFC 064]. - -### RFC 061 `std.compression` - -`std.compression` now provides codec-based compression and decompression for `gzip`, `zlib`, raw `deflate`, `zstd`, `bz2`, XZ/LZMA-family streams, framed `snappy`, and advanced raw Snappy interop. - -```incan -from std.compression import gzip, decompress_auto, Codec - -compressed = gzip.compress(payload)? -codec, plain = decompress_auto(compressed, [Codec.Gzip])? -``` - -Every required codec exposes one-shot byte helpers and stream helpers over `std.io.BytesIO` and `std.fs.File`. Autodetection is decompression-only and opt-in through `decompress_auto(...)` or `decompress_auto_stream(...)`; it uses framing signatures, respects the caller's `allowed` filter exactly, and never guesses from file extensions or MIME types. The public error boundary is `CompressionError`, with stable categories for invalid data, truncated input, unsupported codecs/options, invalid levels, invalid chunk sizes, I/O failures, and backend failures. - -The implementation is dogfooded in Incan stdlib source using ordinary Rust crate imports for the codec boundary rather than new `@rust.extern` function or type implementation surfaces. The generated-project regression fixture covers one-shot, BytesIO stream, file stream, autodetection, option error, and chunk-size error behavior. - -See also: [compression how-to](../language/how-to/compression.md), [std.compression reference](../language/reference/stdlib/compression.md), [RFC 061]. - -## Detailed inventory - -The sections above are the release story. The feature inventory below is separate from stabilization and bugfixes so new surface area can be scanned independently from release hardening. - -### Control-flow features - -- **Language/Compiler**: Incan now supports `loop:` as an explicit infinite-loop construct in both statement and expression position, with `break ` completing the surrounding `loop:` expression and plain `break` remaining valid for `for`, `while`, and statement-form `loop:` (#327, RFC 016). - -### Compiler and code-generation features +## Migrating from 0.2 -- **Compiler/Codegen**: Generic class type-owned factories can now construct and return `Self` from `@classmethod` and `@staticmethod` bodies. The compiler binds `cls(...)` inside classmethods, lowers `Type[T].factory(...)` as a Rust associated call instead of a value-position index expression, and the LSP surfaces `cls` hover/completion inside classmethod bodies (#388). -- **Language/Compiler/Runtime**: RFC 009 implements the numeric type registry with exact-width signed and unsigned integers, pointer-sized integers, `f32`/`f64`, analytics/database aliases including `bigint` and `hugeint`, parameterized `decimal[p, s]` / `numeric[p, s]` literals, lossless numeric widening, explicit integer resize helpers, and exact/lossless Rust interop numeric adaptation (#325, RFC 009). -- **Compiler/Codegen**: RFC 032 value enums now lower their raw-value metadata into IR and generate `value()`, `from_value(...)`, display, string parsing, and serde implementations that use the canonical raw representation while keeping `message()` variant-name based (#317, RFC 032). -- **Compiler/Codegen**: RFC 025 now preserves distinct same-generic-trait instantiations on model, class, and enum declarations, allows trait-backed same-name methods, resolves same-family calls by argument types or explicit expected return type, enforces `T with Trait[F]` generic bound arguments, and emits separate Rust trait impls (#150, RFC 025). -- **Language/Compiler/Codegen**: RFC 043 starts Rust trait implementation authoring from Incan source on newtype and rusttype declarations, using `with TraitName` for adoption, method-level `for TraitName` for same-name method collisions, associated type declarations such as `type Output for Add[int] = UserId`, checked metadata preservation, and generated Rust trait impl emission (#200, RFC 043). -- **Language/Stdlib/Codegen**: RFC 024 adds module-level derive protocols. `std.serde.json` now declares `__derives__ = [Serialize, Deserialize]`, `@derive(json)` adopts both JSON traits, module-qualified bounds such as `T with json.Serialize` typecheck and lower, generated Rust forwards the corresponding serde derives and trait impls, and user-authored derivable modules are covered for both additional Serde-backed formats and pure Incan derivable traits (#148, RFC 024). -- **Language/Compiler**: RFC 017 implements validated newtypes with constrained primitive type syntax such as `int[ge=0]`, canonical `from_underlying` hooks returning `Result[..., ValidationError]`, implicit checked coercion at function arguments, typed initializers, and model/class field construction, fail-fast validation for ordinary coercion sites, aggregated model/class field errors, and `@no_implicit_coercion` opt-outs without adding ambient primitive parsing (#75, RFC 017). -- **Compiler/Codegen**: `@rust.allow(...)` now emits targeted Rust `#[allow(...)]` metadata for specific generated Rust items when an Incan declaration intentionally accepts a narrow rustc or Clippy lint. The decorator supports functions, methods, models, classes, enums, and newtypes, rejects module-level directives, and blocks broad lint groups such as `warnings`, `unused`, and the common Clippy group lints (#337, RFC 057). -- **Language/Compiler**: Enums can now declare methods and associated functions after their variants and adopt traits with `with`, bringing enum-owned behavior and trait protocol participation into parity with models and classes (#334, RFC 050). -- **Language/Compiler**: `match` arms and `if let` patterns now support pattern alternation with `|`, so alternatives such as `Status.Pending | Status.Retrying` can share one branch while still requiring identical binding names and binding types across alternatives (#387, RFC 071). -- **Language/Compiler**: Core syntax now uses statically checked protocol hooks for user-defined truthiness, length, membership, iteration, indexing, indexed assignment, and callable-object invocation (#86, RFC 068). -- **Language/Compiler**: RFC 046 adds computed properties with `property name -> Type:` declarations on models, classes, and trait implementations. Reads use field-like `obj.name` syntax, each read executes the property body, trait properties act as abstract requirements, and property/member name collisions and `obj.name()` calls are diagnosed (#203, RFC 046). -- **Language/Stdlib**: RFC 088 standardizes lazy iterator adapters and terminal consumers on iterator values, including `.batch()` with final partial-batch preservation, `.flat_map()` over `Iterable[U]` callback results, terminal consumption semantics, and `.collect()` returning `list[T]` (#127, RFC 088). -- **Language/Stdlib**: RFC 070 adds Rust-shaped `Result[T, E]` combinators for branch-local transforms, fallible chaining, recovery, and inspection taps: `.map()`, `.map_err()`, `.and_then()`, `.or_else()`, `.inspect()`, and `.inspect_err()` (#386, RFC 070). - -### Tooling and workflow features - -- **Tooling**: RFC 020 completes the Cargo policy contract for generated builds and tests. `incan build`, `incan run`, and `incan test` now accept `--offline`, `--locked`, `--frozen`, explicit `--no-*` environment overrides, Cargo args forwarding, and matching CI environment defaults for restricted-network and reproducible workflows (#38, RFC 020). -- **Tooling**: `incan.lock` files no longer include a volatile generation timestamp. New lock files are deterministic for unchanged dependency inputs, while older lock files with `generated = "..."` metadata remain readable. -- **Tooling**: Default `incan build` and `incan test` now warn and reuse an existing stale `incan.lock` payload instead of rewriting the project lockfile as a side effect of routine verification. `incan lock` remains the explicit refresh command, while `--locked` and `--frozen` keep rejecting stale lockfiles (#446). -- **Tooling**: `incan tools doctor` now includes advisory offline-readiness diagnostics in text and JSON output, reporting Cargo availability, effective Cargo home, cache/config hints, and remediation steps before users rely on RFC 020 offline or frozen policy in restricted environments (#460). -- **Tooling/Editor**: `incan tools doctor` and the VS Code/Cursor **Incan: Doctor** command now report local `incan` / `incan-lsp` path resolution, cargo-bin symlink state, and recovery guidance for stale editor diagnostics or mismatched local binaries (#426). -- **Compiler/Tooling**: RFC 048 checked contract metadata is now compiler-visible through canonical model bundle validation, project materialization, deterministic `incan tools metadata model` emit from projects, bundle JSON, and `.incnlib` artifacts, artifact embedding for publishable bundles, strict checked API docstring validation, `incan tools metadata api` JSON extraction, and LSP hover/emit integration (#205, #438, RFC 048). -- **Compiler/Tooling**: `incan tools metadata api` emits checked public API metadata JSON for an Incan source file or project directory, including public declarations, checked signatures, stable anchors, parsed docstring sections, public import aliases with resolved targets, resolved decorator paths, safe decorator arguments, safe public const values, and model field alias/description metadata (#205, #438). -- **Tooling/Editor**: LSP hover now previews RFC 048 checked API metadata for public declarations and selected public model/class members after successful typechecking, and `workspace/executeCommand` command `incan.metadata.model.emit` emits contract-backed model source or bundle JSON from project, bundle, or artifact metadata (#205). -- **Tooling/Editor**: LSP hover and completion details now surface RFC 032 value-enum metadata. Public value-enum hovers use Incan backing spellings (`str` / `int`), public enum variant hovers show raw values, and local enum/variant completions include backing type and raw-value details (#166, RFC 032). -- **Tooling/Editor**: LSP hover and completion details now include computed property members, showing `property Owner.name -> Type` for model, class, and trait property declarations (#203, RFC 046). -- **Compiler/Tooling**: CLI compilation, LSP dependency collection, and the test runner now share the frontend's canonical source-module resolver for local module paths, logical module identity, stdlib source classification, and source-root fallback behavior (#285). -- **Compiler/Tooling**: RFC 053’s vertical-spacing contract is now reflected in `incan fmt`: top-level `def` / `model` / `type`-like declarations keep two blank lines around them, adjacent constants/statics stay grouped unless they border one of those declarations, trait abstract methods stay tight until a following body-bearing member, docstring indentation is normalized while actual blank-line runs collapse to one blank line, single readability gaps between statement groups survive nested suites, short single-statement `match` arms stay inline, blank lines after suite headers and match-arm arrows are normalized, trailing EOF blank lines are removed, two consecutive blank lines are allowed only at root level, and stand-alone comments attach as leading/trailing bundles even when the formatter wraps the target statement (#336, RFC 053). -- **Compiler/Tooling**: `incan fmt` now wraps overflowing call and constructor argument lists, plus function and method signatures, across multiple lines with trailing commas controlled by the existing formatter setting (#336, #248). -- **Tooling**: Vocab extraction helper tests now reuse the workspace lockfile when resolving helper dependencies, so focused vocab extraction coverage can run in restricted-network environments once local workspace dependencies are present (#211). -- **Tooling/CI**: Downstream Incan projects can now use the repository composite action at `dannys-code-corner/incan/.github/actions/install-incan@` to build the compiler from the pinned repository ref, cache Cargo artifacts, and add the resulting `incan` binary to `PATH` before running project-specific CI commands (#188). -- **Tooling**: Vocab WASM desugarers now get enough fuel to parse, walk, and serialize nested public AST output from real `wasm32-wasip1` companion crates. Regression coverage runs a deeply nested vocab block through `incan run` with a `let` statement whose value contains nested helper-call output, list arguments, action requirements, page interactions, and required-input constraints to guard the desugar boundary reported in #455. -- **Tooling/CI**: Stable Ubuntu, macOS, and MSRV test gates now use sccache-backed nextest slice partitions while preserving the aggregate CI check names, and the release smoke gate uses a dedicated release-profile target cache to reduce duplicated compiler work without dropping broad coverage (#451). -- **Tooling/Test runner**: RFC 019 expands `incan test` with explicit `@test` discovery, stable test ids for `-k` and `--list`, JSON Lines reports with `schema_version: "incan.test.v1"`, JUnit XML output, duration reporting, deterministic shuffle/seed support, `--run-xfail`, conftest inheritance for conventional tests, inline `module tests:` execution, parametrization, fixtures, conditional markers, timeouts, output capture controls, and worker scheduling with `--jobs`, `@resource`, and `@serial` (#77, RFC 019). -- **Tooling/Test runner**: RFC 019 fixture lifecycles now run through worker-batch harnesses, including compatible cross-file session fixture reuse with `--jobs 1`, per-worker session reuse with `--jobs N`, module/session teardown timing, and captured `yield` fixture teardown locals (#77, RFC 019). -- **Tooling/Test runner**: RFC 004 async fixtures now use the existing `@fixture` decorator on `async def`, await setup before dependents run, await post-`yield` teardown, compose with synchronous fixtures under function/module/session scopes, and resolve after parametrized test expansion while keeping timeout policy at the test-batch level (#78, RFC 004). -- **Tooling/Test runner**: Worker batches now fall back to per-file harnesses when multiple source files define colliding top-level Rust item names. Compatible files still batch together for session fixture reuse, while projects with repeated helper/model names avoid generated Rust duplicate-definition failures. -- **Tooling/Test runner**: `incan test` now preheats stale generated Cargo harnesses with `cargo test --no-run`, fingerprints successful preheat state next to each generated harness, and uses a one-writer lock so concurrent CLI/LSP-style runs do not stampede Cargo (#272). -- **Tooling/Test runner**: `incan lock` and implicit first-use lock generation now preheat non-trivial dependency graphs with `cargo test --no-run` into the same debug target domain used by generated test harnesses, then stamp the dependency preheat fingerprint so unchanged relocks stay cheap (#272). -- **Tooling**: Project-aware commands now enforce `[project].requires-incan`, env-level `requires-incan` can narrow named environment workflows, and `incan env show` / `env run --dry-run` report the effective toolchain compatibility before scripts run; RFC 073 matrix expansion remains deferred beyond `0.3` (#401, RFC 073). - -### Language, syntax, and stdlib features - -- **Language/Compiler**: Enum bodies now support same-enum variant aliases such as `WARNING = alias WARN`, letting value enums expose compatibility or readability spellings without creating duplicate raw values or extra runtime variants (#392, RFC 072). -- **Language/Stdlib**: RFC 072 introduces `std.logging` with source-defined `Level`, `Logger`, `LogFormat`, `LogStyle`, `ColorPolicy`, `LogRecord`, `basic_config(...)`, `get_logger(...)`, and the shadowable ambient `log` surface. Logger methods preserve structured primitive and `TelemetryValue` fields, support bound context and child names, infer source-module logger names where metadata exists, and implement filtering plus human/JSON rendering in Incan source. JSON records use `std.telemetry.core` values with OpenTelemetry log data model aliases, `Level.WARN` and `Level.FATAL` are canonical with `WARNING` and `CRITICAL` as aliases, timestamps flow through `std.datetime`, and stdout/stderr delivery uses ordinary `rust::std::io` imports rather than a logging-specific Rust module (#392, RFC 072). -- **Language/Stdlib**: RFC 059 introduces `std.regex` with compiled `Regex` values, `Match` spans, `Captures` groups, safe-default regex semantics, named and indexed capture lookup, explicit `None` for unmatched optional groups, split iterators, first/all/limited replacement, `$1` / `${name}` replacement interpolation, callable replacements, and constructor flags for common modifiers (#294, RFC 059). -- **Language/Stdlib**: `std.graph` adds explicit in-memory graph values with direct `DiGraph[T]()`, `Dag[T]()`, and `MultiDiGraph[T]()` construction, acyclicity-enforcing DAGs, stable `EdgeId` values for parallel multigraph edges, stable `NodeId` values, typed node payloads, node and edge removal, adjacency queries, roots and sinks, BFS/DFS traversal helpers, and cycle-reporting topological order for dependency and plan graphs (#204, RFC 047). -- **Language/Stdlib**: `std.datetime` adds Rust `std::time`-backed `Duration`, `Instant`, and `SystemTime`, plus source-defined Incan civil values for dates, times, naive datetimes, fixed UTC offsets, fixed-offset datetimes, UTC civil clock factories, day/time intervals, year/month intervals, compound datetime intervals, ISO-style parsing/formatting, Python-shaped `strftime` / `strptime` with nanosecond `%f`, deterministic calendar arithmetic, and interval normalization (#292, RFC 058). -- **Language/Stdlib**: `std.collections` adds explicit specialized collection types for double-ended queues, multisets, default-valued maps, ordered maps and sets, sorted maps and sets, layered maps, and priority queues. The namespace is registered as an ordinary source stdlib module with no feature gate, no extra Cargo dependencies, and no Rust-backed stdlib dispatch (#164, RFC 030). -- **Language/Stdlib**: `std.encoding` adds strict-by-default binary-text transform modules for hex, base32, base64, base85, base58, and Bech32/Bech32m, with explicit variant function names, separately named lenient decoders, and source/sink helpers that compose with `std.fs.Path` and `std.io.BytesIO` (#342, RFC 064). -- **Language/Stdlib**: `std.compression` adds codec namespaces for gzip, zlib, raw deflate, zstd, bzip2, XZ/LZMA, framed Snappy, and raw Snappy interop, with source-defined one-shot helpers, stream helpers over `std.fs.File` and `std.io.BytesIO`, explicit decompression autodetection, stable `Codec` and `CompressionError` vocabulary, stdlib-managed generated-project dependencies, and generated-project regression coverage for issue #548 (#339, #548, RFC 061). -- **Language/Compiler**: RFC 029 adds anonymous closed union annotations with canonical `Union[A, B, ...]` and `A | B` syntax. The compiler normalizes duplicates, nested unions, ordering, and `None`-containing unions, accepts member-to-union and union-to-union assignability, lowers ordinary unions to generated closed Rust enums, preserves `None` unions on the existing `Option[...]` path, and supports `isinstance` narrowing for true branches, else branches, wider unions, chained `elif` branches, and `Option[Union[...]]`, plus `is None` / `is not None` narrowing and exhaustive `match` type patterns (#163, RFC 029). -- **Language/Stdlib**: RFC 028 expands `std.traits.ops` into the nominal operator capability vocabulary for custom types, including `FloorDiv`, `Pow`, shifts, bitwise operators, pipe operators, `MatMul`, unary `Not`, `GetItem` / `SetItem`, and explicit in-place compound-assignment traits for `+=`, `-=`, `*=`, `/=`, `//=`, `%=`, `@=`, `&=`, `|=`, `^=`, `<<=`, and `>>=` (#162, RFC 028). -- **Language/Stdlib**: RFC 055 introduces `std.fs` as the path-centric filesystem module: `Path`, `File`, `OpenOptions`, directory entries, metadata, disk usage, structured `IoError`, whole-file byte/text helpers, chunked file handles, traversal, globbing, copy/move, recursive deletion, links, permissions, and explicit durability syncs (#286, RFC 055). -- **Language/Stdlib**: RFC 056 introduces `std.io` for in-memory binary streams with `BytesIO`, `Endian`, cursor helpers, delimiter reads/skips, truncation, buffer extraction, and trait-backed exact-width numeric `read(endian)` / `write(value, endian)` overloads over RFC 009 integer and float types (#291, RFC 056). -- **Language/Compiler**: Incan functions and methods can now declare variadic positional and keyword captures with `*args: T` and `**kwargs: T`, which bind as `List[T]` and `Dict[str, T]` inside the callable. Static call-site unpacking with `f(*xs)` and `f(**kw)` supports rest-aware callees and fixed-parameter callees when the compiler can prove the unpacked shape. Runtime list and dictionary literals now support spread entries with `[*xs]` and `{**kw}`, while invalid destinations such as `[**xs]` and `{*xs}` are rejected with targeted diagnostics (#83, RFC 038). -- **Library authoring**: `incan_vocab` is now versioned as `0.2.0`, marking the first contract bump after the initial 0.1 companion-crate API. The crate README now tracks version history and separates crate semver from the serialized `VOCAB_METADATA_VERSION` and `WASM_DESUGAR_ABI_VERSION` compatibility constants. -- **Language/Compiler**: RFC 040 adds scoped DSL surface descriptors to `incan_vocab` 0.2.0 and library manifests. Imported vocab crates can now publish descriptor metadata for operator-like glyphs, binding-like glyphs, and expression-form surfaces; the parser recognizes descriptor-enabled leading-dot paths and scoped operator glyphs inside owning vocab blocks while preserving ordinary syntax outside those blocks (#174, RFC 040). -- **Language/Compiler**: RFC 036 adds typed user-defined decorators for top-level functions, async functions, and instance methods, including `mut self` methods. Decorators are ordinary callable values applied bottom-up, method decorators receive `&Owner` or `&mut Owner` receiver callables, decorator factories are checked as callable-producing expressions, later references see the post-decoration binding type, and compiler-owned decorators such as `@route`, `@rust.extern`, `@staticmethod`, `@classmethod`, and `@requires` keep their existing special handling (#170, RFC 036). -- **Language/Compiler**: RFC 045 adds scoped DSL symbol descriptors to `incan_vocab` 0.3.0 and library manifests. Imported vocab crates can now publish identifier-call symbols such as `sum(...)` or `count(...)` that resolve as DSL-owned symbols inside eligible vocab positions, prefer innermost owning DSL scope, diagnose active-DSL misuse with descriptor-authored messages, and leave ordinary Incan resolution unchanged outside the DSL scope. Core builtin functions are now explicitly reachable through `std.builtins.` when an unqualified name is shadowed (#202, RFC 045). -- **Language/Compiler**: Incan now supports `if let PATTERN = VALUE:` and `while let PATTERN = VALUE:` for single-pattern control flow. Parsing, formatter round-trips, typechecking, scoping, lowering, and Rust emission now follow the same pattern semantics as `match`, while `if let` stays single-arm only and rejects `else` / `elif` branches in v1 ([RFC 049], #333). -- **Language/Compiler**: Incan now supports RFC 032 value enum declarations with `str` and `int` backing values. The parser and formatter preserve raw variant assignments, while declaration validation rejects missing values, duplicate raw values, mismatched literal types, payload-bearing variants, generated-helper name collisions, and generic value enums (#317, RFC 032). -- **Language/Compiler**: RFC 083 adds declaration-level symbol aliases and same-type method aliases. Top-level forms such as `mean = avg` and `pub average = alias avg` resolve to existing callable or type-like symbols, method aliases such as `mean = avg` project the target method signature without creating wrapper methods, checked API metadata records alias identity, and library manifests now export aliases as alias metadata instead of duplicated declarations (#437, RFC 083). -- **Language/Compiler**: RFC 084 implements callable preset support with RHS partial declarations such as `pub get = partial route(method="GET")`, same-type method partials, trait method partial defaults, local partial expressions, declaration-safe top-level preset values, projected callable signatures where presets display as ordinary defaults, wrapper lowering for top-level function and constructor presets, public manifest and checked-API exports for projected partial signatures, generated Markdown API references for partials, LSP hover/completion/definition/document-symbol support for partials, and diagnostics for unsupported targets, visibility leaks, cycles, rest targets, trait override conflicts, and inherited partial ambiguity (#453, RFC 084). -- **Language/Compiler**: Public classes now preserve authored field visibility. Non-`pub` class fields stay private after formatter round-trips and member access outside the owning class is rejected, while methods on the class can continue to use private backing fields (#246). -- **Compiler/Parser**: Multiline function and method parameter lists now accept a trailing comma before `)`, including receiver-only method signatures such as `def get(self,) -> int` when written across lines (#394). -- **Tooling**: `incan fmt` now wraps long parenthesized logical expression chains at `and` / `or` breakpoints when the inline form exceeds the configured line-length target (#484). -- **Language/Testing**: RFC 018's `assert expr[, msg]` language primitive is always available without importing `std.testing`. The `std.testing` helpers mirror assertion failure behavior for call-style checks, raises checks, and unwrap-style `Option` / `Result` helpers, while marker decorators remain imported `std.testing` APIs. -- **Language/Testing**: Inline `module tests:` blocks in production source files are now discovered and executed by `incan test`, while production build/run output still strips those test-only declarations and imports ([RFC 018], #76). -- **Runtime/Async**: `std.async` now documents cancellation-safety contracts and exposes channel reservation APIs so critical sends can reserve capacity before committing messages (#415, #416). -- **Runtime/Async**: `std.async.time` adds `timeout_join`, `timeout_join_ms`, and a must-use `TimeoutJoinOutcome` so spawned work can keep running after a deadline while callers retain the live `JoinHandle` for later observation or explicit abort (#417). -- **Runtime/Async**: `std.async.sync.Barrier.wait()` now uses Incan-owned generation bookkeeping so cancelling a pending wait withdraws that participant and frees its arrival slot instead of corrupting barrier progress (#418). -- **Language/Compiler**: Async semantic validation now warns when a direct async function or method call is not awaited, and the existing `await`-outside-async type error is routed through the same registry-backed async surface (#146). -- **Language/Compiler**: RFC 044 lets abstract trait methods omit the trailing `: ...` marker while keeping the explicit spelling valid; body-less methods outside traits remain invalid (#201, RFC 044). -- **Language/Runtime/Async**: RFC 039 adds `Awaitable[T]`, expression-position `race for value:` blocks, and the public `std.async.race` helper surface. `std.async.select` is removed rather than kept as a beta-era compatibility alias, `RaceArm`/`arm`/`race` cover helper-style composition, and ready ties resolve in source order (RFC 039, #173). -- **Language/Compiler**: List and dict comprehensions now accept tuple-unpack iteration targets such as `for idx, name in enumerate(xs)`, matching ordinary `for` loop binding syntax (#483). -- **Language/Compiler**: RFC 006 adds lazy `Generator[T]` values, including `yield`-based generator functions, full-clause generator expressions, iteration-protocol compatibility, and the minimum helper surface `.map()`, `.filter()`, `.take()`, and `.collect()` (#324, RFC 006). -- **Language/Stdlib**: RFC 069 adds import-free `list.repeat(value, count)` for fixed-length list initialization. The compiler infers `list[T]`, enforces clone-compatible repeated values and `count: int`, lowers recognized calls to the stdlib helper, and raises `ValueError` with the bad count for negative runtime counts (#385, RFC 069). -- **Language/Stdlib**: `std.uuid` adds source-defined UUID values with parsing, canonical formatting, `u128` and RFC/network-order byte conversion, nil/max and namespace constants, version/variant inspection, and generation helpers for UUID versions 1, 3, 4, 5, 6, 7, and 8 while keeping UUID layout semantics in Incan source (#338, RFC 060). -- **Language/Compiler**: `List[T].clone()` now typechecks when `T` satisfies `Clone`, returns `List[T]`, and emits the same element-cloning container copy that Rust `Vec::clone()` provides (#363). -- **Language/Interop**: Direct `list[T]` arguments passed to external Rust functions or methods can now satisfy `Vec` parameters by mapping elements through Rust `.into()` at the call boundary, covering APIs such as Polars constructors that accept `Vec` from `Series` values (#128). - -## Stabilization and bugfixes - -- **Compiler/Codegen**: Duckborrowing ownership planning is now centralized around typed value-use sites, covering Incan call arguments, Rust interop arguments, struct fields, collection and tuple elements, assignments, returns, match scrutinees, mutable aggregate parameters, collection lookup probes, loop/comprehension traversal, and backend-inserted generic `Clone` bounds. This removes several classes of generated-Rust borrow/move failures and reduces the need for user-authored `.clone()`, `.as_ref()`, `str(...)`, and `.into()` workarounds (#121). -- **Compiler/Codegen**: Default argument expressions that call helpers imported into the defining module now emit those helper calls with the required module qualification when the default is expanded at another call site. This fixes generated Rust failures such as omitted defaults expanding to an unqualified `fallback()` in test runners or downstream modules (#395). -- **Compiler/Codegen**: Ordinary anonymous union wrappers are now shared through the generated crate root for multi-file source modules, so same-shaped unions can be forwarded across modules and member literals can call imported union-typed functions without producing distinct or unqualified Rust wrapper types (#457, #461). -- **Compiler/Codegen**: Wide ordinary-union `isinstance` chains now fully lower before Rust emission, preserving the documented chained narrowing surface instead of leaving runtime `isinstance(...)` calls in generated Rust (#458). -- **Compiler/Codegen**: Generated Rust now retains Rust enum imports that are referenced only from match patterns, including prost-style patterns such as `Some(RelType.Read(_))` (#459). -- **Compiler/Codegen**: `std.testing.assert_eq` and `assert_ne` now isolate their generated Rust operands before comparing them, so checks such as `assert_eq(plan_encoded_len(plan) > 0, true)` emit valid Rust instead of a chained-comparison parse error. -- **Compiler/Codegen**: Cross-module trait-bound propagation no longer lets a same-named external generic helper rewrite a local non-generic function signature. This keeps `std.testing.timeout(...)` independent from `std.async.time.timeout(...)` even though both helpers share the same leaf name. -- **Compiler/Codegen**: Normal generated Rust no longer emits compiler-generated `dead_code` or `unused_imports` allowances. The backend now prunes unused private declarations and imports, keeps Rust extension-trait imports when method lookup needs them, keeps public reexports warning-clean without suppression, and uses narrow `#[expect(dead_code)]` markers only where retained private fields are required for Incan semantics but Rust cannot observe a read (#214). -- **Compiler/Runtime**: Generated Rust now routes the in-scope panic-backed collection and JSON extraction paths, plus proc-macro decorator misuse stubs, through named stdlib helpers instead of open-coded fallback or `panic!` shims. The narrow checked-newtype construction panic remains tracked separately (#351). -- **Compiler/Runtime**: Generated project manifests now keep Tokio and `serde_json` behind the corresponding `incan_stdlib` feature gates for ordinary async and JSON stdlib use, reducing direct generated `Cargo.toml` dependencies without changing the public `std.async` or `std.serde.json` APIs (#157). -- **Compiler/Typechecker**: Typechecker architecture is now split across clearer internal ownership boundaries. Lowering-facing semantic snapshots live outside the main checker state, stdlib trait-method fallback lookup comes from the canonical stdlib registry surface, and import materialization is decomposed into explicit module, stdlib, pub, and Rust import paths without changing language behavior (#283). -- **Compiler/Typechecker**: Unsupported trait-typed local annotations now produce an Incan diagnostic instead of reaching Rust codegen as invalid bare trait local types (#462). -- **Language/Compiler**: Multi-file web builds now retain private route-decorated handlers and the private models they use in dependency modules, so route registration works without forcing those declarations public (#117). -- **Language/Compiler**: Stdlib import validation now rejects unknown names from known stdlib modules, imported stdlib static method calls preserve default arguments at the call site, union narrowing lowers chained `isinstance` branches without leaking raw `isinstance` calls or unit fallthroughs into generated Rust, and Rust interop accepts owned Incan values for shared borrowed generic parameters such as `&T` in both free-function and method-call positions (#499, #500, #501, #502, #506, #508). -- **Language/Interop**: Generated Rust now retains extension-trait imports from typechecker import metadata and receiver trait metadata instead of backend method-name heuristics, so same-name methods from unrelated imported traits do not force unused trait imports (#447). -- **Language/Interop**: Metadata-backed external Rust calls now preserve inspected generic by-value parameters, so prost-style inherent and trait-provided `decode(buf: T)` calls pass owned cursors or borrowed slices directly instead of generating an invalid shared borrow (#609, #612). -- **Tooling/Compiler**: `incan test` now includes implicit generated-code stdlib helper modules such as `std.result` when test files use helper-backed surfaces such as `Result.map_err`, matching the build/check/run dependency closure (#610). -- **Tooling**: `incan lock` now treats manifest projects as a project-wide lock surface, covering declared scripts and test harness dependency inputs so multi-entrypoint projects do not alternate stale-lock warnings between `incan test` and `incan run src/extra.incn` (#505). -- **Tooling**: `incan fmt` now wraps long class trait adoption headers into parseable parenthesized `with (...)` lists, keeping broad adoption surfaces such as `_BytesIO` readable and below the line-length target (#565). -- **Tooling/Compiler**: Generated Rust quality now has artifact-level package baselines, representative stdlib generated-Rust snapshots plus coverage inventory, an audit-report helper with a deterministic strict gate, package-facing callable characterization, a native Rust consumer fixture for generated libraries, and ownership-planner hot-path improvements that avoid proven-unnecessary clone calls for Copy comprehensions and selected owned iterator sources (#599, #600, #601, #602, #603). - -## Documentation and release hardening - -- **Docs**: Added explanation pages for compile-time vs runtime behavior and Rust-shaped confidence, with navigation links and evaluator-guide cross-links for Python and Rust users. -- **Docs**: Added a binary-text encoding how-to for choosing `std.encoding` formats, strict decoding at boundaries, stream/path transforms, and Bech32 five-bit payload handling. -- **Docs**: Stdlib reference pages now keep API contracts separate from task guidance: `std.graph`, `std.regex`, `std.logging`, and `std.hash` link to dedicated how-to or explanation pages, while existing UUID, tempfile, collections, encoding, compression, and datetime references were trimmed back toward reference material. -- **Contributor docs**: Workspace crate boundaries are now classified as stable contracts, compiler/toolchain implementation, runtime-only implementation, and transitional runtime surfaces. The docs also call out explicit ownership metadata for shared surface types, staged Rust interop inspection, and the quarantined `std.web` host-runtime bridge (#284). - -### Versioning and release track - -- **Project lifecycle tooling**: Added lifecycle commands for interactive `incan new` / `incan init`, `incan version`, and `incan env`, plus project lifecycle documentation and `incan.toml` environment metadata support (#73). -- **Dependency policy**: The rust-analyzer proc-macro API dependency is patched locally to request `postcard` without default features, removing the unmaintained `atomic-polyfill` crate from the workspace dependency graph and letting `cargo deny check` run without the `RUSTSEC-2023-0089` advisory ignore (#260). -- **Dependency policy**: The workspace now builds against Wasmtime `44.0.1` / Wasmtime WASI `44.0.1` and raises the Rust MSRV to `1.92`, matching Wasmtime 44's compiler requirement. -- **Dependency policy**: Dependabot security alerts for the VS Code extension lockfile, docs-site Python pins, and Rust `rand` lock entries are remediated, while repo-owned GitHub Actions are moved to Node 24-compatible action releases (#475, #464). -- **Release inventory**: The release-note inventory was reconciled against the 0.3 milestone closeout. Theme-level bullets above cover the detailed generated-Rust, formatter, dependency, test-runner, Rust interop, stdlib, lifecycle, and RFC implementation work; release-relevant direct references include #607, #605, #604, #571, #562, #547, #492, #488, #414, #343, #335, #322, #280, #262, #241, #222, #149, #131, #82, #80, #79, #74, #70, and #69 where those items are grouped rather than named as standalone headline bullets. - -## Known limitations (0.3) - -- Decimal arithmetic is not yet general language behavior. The RFC 009 decimal surface covers typed annotations, literal validation, formatting, generated Rust representation, and display; arithmetic semantics should wait for a follow-up language/library decision. -- `incan fmt` remains intentionally conservative on broader wrapping and may still leave indivisible tokens or unsupported expression shapes beyond the documented 120-character line-length target. RFC 053 / #336 narrows the vertical-spacing contract and adds call/constructor wrapping, while #248 adds common function/method signature wrapping; this is still not a general wrapping/configuration overhaul. -- `std.regex` is the safe default regex surface, not a Python/PCRE compatibility layer. Lookaround, pattern backreferences, and other backtracking-only features belong in a separate package track if they are standardized later. -- Native Windows filesystem semantics are not part of the 0.3 contract. The `std.fs` surface is documented for Unix-like host behavior until the stdlib grows an explicit platform split. +Most ordinary `0.2` programs should continue to compile. The changes below are the ones most likely to show up during adoption. + +1. **Formatter output may change.** `incan fmt` now follows RFC 053's vertical-spacing buckets and wraps more calls/signatures. Projects that run `incan fmt --check` should expect one intentional formatting diff. +2. **Numeric names are more reserved.** Existing `int` and `float` code keeps working, but project-local bare type names such as `decimal`, `numeric`, `bigint`, `integer`, `smallint`, `real`, or `double` can now collide with canonical numeric vocabulary. Rename local aliases or use the new exact forms such as `decimal[12, 2]`. +3. **Testing imports are clearer.** The language `assert` statement is always available, but testing decorators and helpers remain `std.testing` APIs. Files that use `@fixture`, `@parametrize`, `@skip`, `assert_eq`, or similar helpers should import them explicitly. +4. **Lockfiles are less noisy.** `incan.lock` no longer records generation timestamps, and routine `build` / `test` runs warn and reuse stale lock payloads instead of rewriting committed lockfiles. Run `incan lock` when you intentionally refresh the lock. +5. **New features are additive.** `loop:`, `if let`, `while let`, value enums, protocol hooks, iterator adapters, and `Result` combinators do not require rewriting existing `match`, `while True`, helper-function, or nested-`match` code. + +## Feature guide + +Use this section as the map. The release note names each larger feature, says what it is for, and links to the docs that carry the real detail. + +### Language features + +- **Numeric types and fixed-scale decimals**: Use exact widths and schema-shaped names when a boundary needs them, while keeping `int` and `float` ergonomic for ordinary code. Start with [Choosing numeric types](../language/how-to/choosing_numeric_types.md), then [Numeric semantics](../language/reference/numeric_semantics.md) ([RFC 009], #325). +- **Loop expressions**: Use `loop:` plus `break ` for search, retry, and accumulator-free loops that produce a value. Read [Control Flow](../language/explanation/control_flow.md) ([RFC 016], #327, #387). +- **Pattern control flow**: Use `if let` and `while let` when one successful pattern should run and the miss case should do nothing. Read [Control Flow](../language/explanation/control_flow.md) ([RFC 049], #333). +- **Union narrowing and pattern alternation**: Model inputs that can take several shapes, then narrow them with checked patterns instead of hand-written tag logic. Read [Union types](../language/reference/union_types.md) ([RFC 071], [RFC 029]). +- **Value enums**: Keep enum type safety while exposing canonical `str` or `int` representations for external values. Read [Enums](../language/explanation/enums.md) and [Modeling with enums](../language/how-to/modeling_with_enums.md) ([RFC 032], #317). +- **Enum methods and trait adoption**: Put enum-owned behavior on the enum and let enums adopt the same trait protocols as other source types. Read [Enums](../language/explanation/enums.md) and [Traits as language hooks](../language/explanation/traits_as_language_hooks.md) ([RFC 050], #334). +- **Computed properties and protocol hooks**: Define property-like readers and dunder-backed operator/protocol behavior without pushing users into Rust-shaped wrappers. Read [Traits as language hooks](../language/explanation/traits_as_language_hooks.md) and [Derives and traits](../language/reference/derives_and_traits.md) ([RFC 046], [RFC 068], [RFC 028], #86, #162, #203). +- **Decorators**: Typecheck user-defined decorators for functions, async functions, and methods so later references see the decorated callable shape, concrete decorated callable values expose `__name__` for registry-style decorators, and decorated wrappers preserve source default-argument calls when the callable surface is unchanged. Read [Decorators](../language/reference/language.md#decorators), [Functions](../language/reference/functions.md), and [Checked API metadata](../tooling/reference/checked_api_metadata.md) ([RFC 036], #170, #640, #694, #703). +- **Symbol aliases**: Export an existing callable or type-like symbol under another name without pretending it is a hand-written wrapper. Read [Symbol aliases](../language/reference/symbol_aliases.md) ([RFC 083], #437). +- **Callable presets with RHS `partial` declarations**: Write `pub get = partial route(method="GET")` when a new API name is really the same callable with named defaults, not a new function body. Read [Callable presets explained](../language/explanation/callable_presets.md), then [Callable presets](../language/reference/callable_presets.md) ([RFC 084], #453). +- **Variadics and call unpacking**: Describe call shapes that accept or forward flexible argument lists without losing static checks. Read [Functions](../language/reference/functions.md) ([RFC 038], #83). +- **Generators and lazy iterators**: Build pipelines with generator values, lazy adapters, and explicit terminal consumers such as `collect`, `count`, `find`, and `fold`. Read [Generators](../language/how-to/generators.md) and [Generator semantics](../language/explanation/generators.md) ([RFC 006], [RFC 088], #127, #324, #386). + +### Rust interop and API metadata + +- **Rust trait adoption from Incan source**: Newtypes and rusttypes can adopt Rust traits with `with Trait`, method-level `for Trait`, and associated type declarations. Read [Rust interop](../language/how-to/rust_interop.md), [Rust types for Python developers](../language/how-to/rust_types_for_python_devs.md), and [`std.traits`](../language/reference/stdlib/traits.md) ([RFC 043], #200). +- **Derived and inspected Rust metadata**: Supported `@rust.derive(...)`, associated types, inspected Rust signatures, and metadata-backed call boundaries now survive further through generated calls. Read [Derives and traits](../language/explanation/derives_and_traits.md) and [Rust-shaped confidence](../language/explanation/rust_shaped_confidence.md) ([RFC 041], #175). +- **Checked public API metadata**: Public declarations, aliases, partials, models, enum variants, decorator-backed callable context, and facade alias projections can be emitted for tools and downstream consumers. Read [Checked API metadata](../tooling/reference/checked_api_metadata.md) and [LSP protocol support](../tooling/reference/lsp_protocol_support.md) ([RFC 048], #205, #438, #694, #695). + +### Standard Library + +- **[`std.async`](../language/reference/stdlib/async.md)**: Awaitable races, channel reservation, timeout joins, cancellation-safe barriers, and un-awaited-call diagnostics move async workflows into documented stdlib APIs ([RFC 039], #173, #415, #416, #417, #418, #146). +- **[`std.collections`](../language/reference/stdlib/collections.md)**: Ordered, sorted, counter, queue, stack, multimap, bidict, and `list.repeat(value, count)` workflows have first-party containers and helpers; see also [Choosing collection types](../language/how-to/choosing_collections.md) ([RFC 030], [RFC 069], #385). +- **[`std.collections.OrdinalMap`](../language/explanation/ordinal_map.md)**: Deterministic immutable key-to-ordinal lookup supports schemas, catalogs, dictionary-encoded domains, and reproducible serialized lookup tables ([RFC 101]). +- **[`std.compression`](../language/reference/stdlib/compression.md)**: Gzip, zlib, deflate, bzip2, lzma, zstd, and Snappy-oriented byte/stream workflows are codec-explicit; see [Compression](../language/how-to/compression.md) ([RFC 061]). +- **[`std.datetime`](../language/reference/stdlib/datetime.md)**: Dates, times, datetimes, durations, parsing, formatting, clocks, and timezone offsets share one temporal vocabulary; see [Dates and times](../language/how-to/dates_and_times.md) ([RFC 058]). +- **[`std.encoding`](../language/reference/stdlib/encoding.md)**: Base64, hex, URL, and related byte/text transforms are strict and named; see [Binary-text encoding](../language/how-to/binary_text_encoding.md) ([RFC 064]). +- **[`std.fs`](../language/reference/stdlib/fs.md)**: Path-centric filesystem work covers paths, metadata, directory traversal, and file operations; see [File I/O](../language/how-to/file_io.md) ([RFC 055]). +- **[`std.graph`](../language/reference/stdlib/graph.md)**: Directed graph, DAG, traversal, dependency ordering, path query, and cycle-aware workflows are available without ad hoc containers; see [Working with graphs](../language/how-to/working_with_graphs.md) and [Graph model](../language/explanation/graph_model.md) ([RFC 047]). +- **[`std.hash`](../language/reference/stdlib/hash.md)**: Byte, file, and reader hashing use algorithm-specific helpers with normalized digest output; see [Hashing data](../language/how-to/hashing_data.md) ([RFC 065]). +- **[`std.io`](../language/reference/stdlib/io.md)**: In-memory byte streams and buffered readers cover byte-oriented I/O without direct Rust interop ([RFC 056]). +- **[`std.json`](../language/reference/stdlib/json.md)**: `JsonValue` supports dynamic payload construction, inspection, conversion, and extraction at API boundaries; see [Dynamic JSON](../language/how-to/dynamic_json.md) ([RFC 051]). +- **[`std.logging`](../language/reference/stdlib/logging.md)**: Structured logging gives modules stable logger names, levels, fields, and runtime-friendly generated Rust output; see [Logging](../language/how-to/logging.md) ([RFC 072]). +- **[`std.regex`](../language/reference/stdlib/regex.md)**: Safe-default regular expressions cover matching, captures, iteration, splitting, and replacement without backtracking-only features; see [Regular expressions](../language/how-to/regular_expressions.md) ([RFC 059]). +- **[`std.result`](../language/reference/stdlib/result.md)**: `Result[T, E]` gained Rust-shaped `map`, `map_err`, `and_then`, `or_else`, `inspect`, and `inspect_err` helpers for fallible pipelines ([RFC 070], #386). +- **[`std.tempfile`](../language/reference/stdlib/tempfile.md)**: Scoped temporary files and directories are first-party test and application resources ([RFC 010]). +- **[`std.testing`](../language/reference/stdlib/testing.md)**: Fixtures, parametrization, markers, temp/env fixtures, async fixtures, and assertion helpers back the `incan test` workflow; see [Testing in Incan](../language/how-to/testing_stdlib.md) ([RFC 018], [RFC 019], [RFC 004], #76). +- **[`std.uuid`](../language/reference/stdlib/uuid.md)**: UUID parsing, formatting, generation, version inspection, and byte/string conversion are available as source-defined helpers; see [Working with UUIDs](../language/how-to/working_with_uuids.md) ([RFC 060]). + +### Tooling + +- **`incan test`**: Inline `module tests:` blocks are discovered by the runner, with fixtures, parametrization, markers, resource-aware parallelism, JSON Lines, JUnit XML, durations, shuffling, and `--nocapture`; read [Tooling: Testing](../tooling/how-to/testing.md) and [Testing in Incan](../language/how-to/testing_stdlib.md) ([RFC 018], [RFC 019], [RFC 004], #76). +- **`incan fmt`**: Formatting follows the vertical-spacing contract and wraps more long calls and signatures; read [Formatting with `incan fmt`](../tooling/how-to/formatting.md) and the [Code Style Guide](../language/reference/code_style.md) ([RFC 053], #73). +- **Cargo policy and lockfiles**: `incan build`, `incan run`, and `incan test` propagate offline, locked, and frozen policy while `incan lock` owns intentional lock refreshes; read [Project configuration](../tooling/reference/project_configuration.md) ([RFC 020], #460). +- **Lifecycle and diagnostics**: `incan new`, `incan init`, `incan version`, `incan env`, and `incan tools doctor` cover project startup, environment inspection, offline readiness, and editor binary health; read [Project lifecycle](../language/how-to/project_lifecycle.md) ([RFC 015], #426). + +## Bugfixes and Hardening + +This section is grouped by outcome rather than by every minimized repro. Issue numbers are kept for traceability when you need the exact bug report. + +### Compiler Correctness + +- **Argument planning is shared**: Ownership and Rust-boundary coercion now route through one argument-use plan instead of parallel caller/emitter heuristics. +- **Duckborrowing covers more real use sites**: Generated Rust handles arguments, returns, assignments, match scrutinees, aggregate elements, lookups, comprehensions, mutable aggregate parameters, Rust calls, and generated `Clone` bounds with fewer user-authored workarounds (#121, #241, #364, #366, #367, #372, #383, #391, #602). +- **Release smoke paths are less fragile**: InQL and release smoke testing fixed loop-item fields, union call arguments, storage-rooted match scrutinees, static list index assignment, typed `assert false` lowering, and const model metadata constructors (#620, #621, #622, #627, #630, #644, #671, #674). +- **Generic and trait flow keeps more type information**: Instantiated receivers, generic fields, generic methods, `list[Self]`, trait/supertrait upcasts, imported prost oneofs, explicit generic cycles, and static factory locals now survive typechecking and lowering more consistently (#237, #231, #253, #230, #184, #218, #279, #252, #255). +- **Runtime-boundary errors are clearer**: `std.regex` text borrowing, collection f-string formatting, and collection/string conversion diagnostics fail closer to the Incan source instead of surfacing as obscure Rust/runtime errors (#624, #625, #71, #81). +- **Stringly compiler behavior is guarded**: Remaining semantic string comparisons in high-risk compiler paths are fingerprinted so new string-based behavior has to be centralized or explicitly classified. + +### Rust Interop And Generated Rust + +- **Borrowing decisions are metadata-backed**: Borrowed `str`/bytes calls, method fallback borrowing, and retained generated imports now follow the same decisions across typechecking, lowering, and emission. +- **Rust bridge identity is preserved**: Inspected methods with unknown generic or lifetime placeholders and re-exported Rust argument displays keep stable bridge identity (#645, #630). +- **Collection adaptation is less manual**: Owned Incan values can flow to shared borrowed generic Rust parameters, and `list[T]` can adapt to `Vec` where metadata proves the boundary (#506, #128). +- **Protobuf-style APIs need fewer workarounds**: Prost-style inherent and trait-provided `decode(buf: T)` calls lower correctly (#609, #612). +- **Generated Rust pruning is safer**: Enum-pattern imports and metadata-derived extension-trait imports survive pruning, while unused generated Rust is pruned without broad `allow` attributes (#459, #447, #214). +- **Generated manifests stay smaller**: Tokio and `serde_json` stay behind feature gates, and generated helper stubs use named helpers (#351, #157). +- **Trait annotation failures are Incan diagnostics**: Trait-typed local annotations now produce diagnostics instead of obscure lowering or generated-Rust failures (#462). + +### Multi-File And Packages + +- **Cross-module codegen is more predictable**: Imported defaults qualify correctly, same-shaped union wrappers are shared, wide union narrowing lowers fully, keyword-named modules escape consistently, and public submodule reexports work under `src/` (#395, #457, #461, #458, #122, #287). +- **Web registration keeps private internals private**: Private route handlers and models are retained for web registration without making them user-visible public API (#117). +- **Package exports match ordinary builds**: Public aliases, public partial presets, package-boundary alias consumption, lowercase exported statics, imported static decorator strings, and keyword-named public symbols follow the same rules across build modes (#617, #631, #633, #658, #659, #698). +- **Partial presets keep their defaults in decorators**: Imported public partials now retain their projected default arguments and module-owned default symbols when used inside decorator factory arguments, matching ordinary runtime calls (#698, #701). +- **Decorator metadata crosses package boundaries**: Source signatures, imported/decorator `const str` arguments, generic decorator factories, method-call decorator factories, and reexport-only facade projections are represented in checked metadata more reliably (#636, #638, #640, #669, #694, #695). +- **Decorator helpers can inspect generic callables**: `func.__name__` works in generic `(F) -> F` decorator helpers, including imported alias and union callable signatures, so registry decorators can infer the decorated helper name instead of repeating it as a string (#694, #701). +- **Decorated wrappers preserve defaults**: Decorated functions and methods keep source default-argument call behavior when the final decorated callable surface still matches the original declaration, including direct imports and public facade re-exports (#703). +- **Script and test manifests are scoped**: Generated Cargo manifests include only reachable dependencies instead of blindly inheriting package-level heavy dependencies (#665). + +### Formatter And Test Runner + +- **Formatter output preserves meaning**: Tuple-unpack comprehensions, f-string debug markers, escaped f-string newlines, numeric spelling, qualified enum/constructor patterns, `mut`, docstrings, trailing commas, logical-expression wrapping, and class trait-adoption wrapping now round-trip more safely (#615, #616, #235, #250, #264, #289, #247, #394, #484, #565). +- **Comprehension behavior is more complete**: `?` propagation works inside comprehensions, and collection/string conversion diagnostics are clearer when runtime coercion fails. +- **Inline tests keep file-local scope**: Directory inline-test runs preserve each file's parser and import scope, conventional test batches split on imported-name collisions, and decorated functions named like builtins resolve to the source binding inside inline tests (#676, #677). +- **The test harness reuses more correctly**: `incan test` reuses generated harness state, isolates single-file runs, keeps project cwd stable, and includes helper modules such as `std.result` when test files use helper-backed surfaces (#268, #269, #271, #288, #378, #610, #505). + +### Docs And Dependencies + +- **User docs are closer to Divio shape**: Stdlib pages for graph, regex, logging, hash, UUID, tempfile, collections, encoding, compression, datetime, and related modules now separate reference contracts from how-to or explanation material (#284). +- **Contributor docs name the important boundaries**: Crate boundaries, ownership metadata, staged Rust inspection, and the quarantined `std.web` host-runtime bridge are documented for maintainers (#284). +- **Dependency alerts are closed out**: The `atomic-polyfill` advisory path is removed, Wasmtime/WASI and MSRV move together, Dependabot alerts are remediated across docs-site Python pins, VS Code lockfiles, Rust lock entries, and GitHub Actions, and `pymdown-extensions` is pinned to `10.21.3` for `GHSA-62q4-447f-wv8h` (#260, #475, #464). + +## Known limitations + +- Decimal arithmetic is not yet general language behavior. The `0.3` decimal surface covers typed annotations, literal validation, formatting, generated Rust representation, and display; arithmetic semantics need a follow-up language/library decision. +- `incan fmt` is still conservative. RFC 053 gives vertical spacing and common wrapping rules, but it is not a general pretty-printer overhaul for every nested expression shape. +- `std.regex` is a safe-default regular-expression surface, not a Python/PCRE compatibility layer. Lookaround, pattern backreferences, and other backtracking-only features are tracked separately by [RFC 100] for a future `std.re` surface. +- Native Windows filesystem behavior is not part of the `0.3` contract. `std.fs` documents Unix-like host behavior until the stdlib has an explicit platform split. ## RFCs implemented -- Async fixtures: [RFC 004](../RFCs/closed/implemented/004_async_fixtures.md) -- Numeric type system and builtin type registry: [RFC 009] -- Temporary files and directories: [RFC 010] -- Hatch-like tooling and project lifecycle CLI: [RFC 015] -- Loop expressions and break values: [RFC 016] -- Validated newtypes with implicit coercion: [RFC 017](../RFCs/closed/implemented/017_validated_newtypes_with_implicit_coercion.md) -- Testing language primitives: [RFC 018] -- Extensible derive protocol: [RFC 024] -- Trait-based operator overloading: [RFC 028] -- Union types and type narrowing: [RFC 029] -- Extended collection types: [RFC 030] -- Value enums: [RFC 032] -- Variadic positional arguments and keyword capture: [RFC 038] -- Open-ended trait methods: [RFC 044] -- Async race and awaitability: [RFC 039](../RFCs/closed/implemented/039_race_for_awaitable_concurrency.md) -- Computed properties: [RFC 046] -- Checked contract metadata and interrogation tooling: [RFC 048] -- Lightweight directed graph types: [RFC 047] -- `if let` and `while let` pattern control flow: [RFC 049] -- Enum methods and enum trait adoption: [RFC 050] -- Dynamic JSON values: [RFC 051] -- Formatter vertical spacing buckets: [RFC 053] -- Path-centric filesystem APIs: [RFC 055] -- `std.datetime` temporal values and intervals: [RFC 058] -- `std.regex` regular expressions, captures, splitting, and replacement: [RFC 059](../RFCs/closed/implemented/059_std_regex.md) -- UUID parsing, formatting, inspection, and generation: [RFC 060](../RFCs/closed/implemented/060_std_uuid.md) -- Codec-based compression and decompression: [RFC 061](../RFCs/closed/implemented/061_std_compression.md) -- Binary-text encoding and decoding utilities: [RFC 064] -- Byte, file, reader, cryptographic, compatibility, and non-cryptographic hashing: [RFC 065] -- Targeted generated-Rust lint suppression: [RFC 057] -- Protocol hooks for core syntax: [RFC 068] -- Fixed-length list initialization with `list.repeat`: [RFC 069] -- Result combinators: [RFC 070](../RFCs/closed/implemented/070_result_combinators_for_result_types.md) -- `std.collections.OrdinalMap` deterministic ordinal indexes: [RFC 101](../RFCs/closed/implemented/101_std_collections_ordinal_map.md) +### Language and compiler + +- [RFC 004]: async fixtures +- [RFC 006]: Python-style generators +- [RFC 009]: numeric type system and builtin type registry +- [RFC 016]: `loop` and `break ` loop expressions +- [RFC 017]: validated newtypes with implicit coercion +- [RFC 018]: language primitives for testing +- [RFC 024]: extensible derive protocol +- [RFC 025]: multi-instantiation trait dispatch +- [RFC 028]: trait-based operator overloading +- [RFC 029]: union types and type narrowing +- [RFC 032]: value enums with `str` and `int` backing values +- [RFC 036]: user-defined decorators +- [RFC 038]: variadic args and unpacking +- [RFC 039]: `race` for awaitable concurrency +- [RFC 043]: Rust trait implementation from Incan +- [RFC 044]: open-ended trait methods +- [RFC 046]: computed properties +- [RFC 049]: `if let` and `while let` pattern control flow +- [RFC 050]: enum methods and enum trait adoption +- [RFC 053]: formatter vertical spacing buckets +- [RFC 057]: targeted Rust lint suppression for generated code +- [RFC 068]: protocol hooks for core language syntax +- [RFC 069]: `list.repeat` helper for fixed-length initialization +- [RFC 070]: result combinators for `Result[T, E]` +- [RFC 071]: pattern alternation in `match` and `if let` +- [RFC 083]: symbol and method aliases +- [RFC 084]: RHS partial callable presets +- [RFC 088]: iterator adapter surface + +### Standard library + +- [RFC 010]: Python-style `tempfile` standard library +- [RFC 030]: `std.collections` extended collection types +- [RFC 047]: lightweight directed graph types +- [RFC 051]: `JsonValue` for `std.json` +- [RFC 055]: `std.fs` filesystem APIs +- [RFC 056]: `std.io` byte streams and binary parsing helpers +- [RFC 058]: `std.datetime` temporal values, intervals, and runtime timing +- [RFC 059]: `std.regex` regular expressions, matches, captures, and replacement +- [RFC 060]: `std.uuid` parsing, generation, and formatting +- [RFC 061]: `std.compression` codec-based compression and decompression +- [RFC 064]: `std.encoding` binary-text encoding and decoding utilities +- [RFC 065]: `std.hash` stable hashing primitives +- [RFC 072]: `std.logging` logger acquisition, configuration, and structured events +- [RFC 101]: `std.collections.OrdinalMap` deterministic key-to-ordinal lookup + +### Tooling and metadata + +- [RFC 015]: hatch-like project lifecycle tooling +- [RFC 019]: test runner, CLI, and ecosystem +- [RFC 020]: offline, locked, and reproducible builds +- [RFC 040]: scoped DSL surface forms +- [RFC 045]: scoped DSL symbol surfaces +- [RFC 048]: checked contract metadata, Incan emit, and interrogation tooling --8<-- "_snippets/rfcs_refs.md" diff --git a/workspaces/docs-site/docs/roadmap.md b/workspaces/docs-site/docs/roadmap.md index 52f4a08b9..29d330320 100644 --- a/workspaces/docs-site/docs/roadmap.md +++ b/workspaces/docs-site/docs/roadmap.md @@ -1,11 +1,12 @@ -# Incan Roadmap (Status-Focused) +# Incan Roadmap -This page tracks the implementation status and near-term planning (without being prescriptive about timelines). +This page tracks implementation status, release scope, and sequencing. Incan development is driven by RFCs (Request for Comments). - An RFC captures a design proposal for a feature, including syntax, semantics, and implementation details. - RFCs are not necessarily implemented in the order they are written. +- Milestones track release posture and sequencing. They define scope, not urgency. See the [RFCs](RFCs/index.md) page for more information about RFCs. @@ -15,62 +16,192 @@ This table is autogenerated from the RFC files (it reads each RFC’s `**Status: --8<-- "_snippets/tables/rfcs_index.md" -## Core Phases (overview) +## Strategic Direction -- Core language + runtime -- Stdlib + tooling (fmt, test, LSP, VS Code extensions) -- Web backend (Axum) -- Interactive runtime stdlib contracts (target manifests, host capabilities, artifacts, optional GPU surfaces) — [RFC 092](RFCs/092_interactive_runtime_stdlib_contracts.md) -- Rust interop +Incan's current direction is: -## Current Focus +> Python-readable at the base, domain-native at the edges, compiler-inspectable all the way down. -- Language stability/feature freeze (core semantics + test surface): - - [RFC 000] (core semantics) *Done* - - [RFC 008] (const bindings) *Done* - - Tests surface: - - [RFC 001] (test fixtures) *In Progress* - - [RFC 002] (parametrized tests) *Draft* - - [RFC 004] (async fixtures) *Done* -- Interactive runtime stdlib contracts ([RFC 092]): **Draft** — target manifests, host capability declarations, execution regions, artifact metadata, diagnostics, input/accessibility hooks, and optional GPU capability surfaces for downstream runtime consumers +That means Incan should not compete as a small systems language or as a generic Python clone. The compiler, standard library, and tooling should make domain packages, capability metadata, policy, generated artifacts, diagnostics, and backend facts inspectable by humans and agents. -## Ecosystem keystones (planned) +The near-term roadmap is therefore split into six release lanes: -These are the cross-cutting capabilities that make Incan feel “capable” for real engineering work. This list is intentionally kept high-level and status-oriented (RFCs will be added over time). +- Tooling and first-contact inspection. +- Backend replacement foundation. +- Backend cutover. +- Broader feature reopening after the compiler architecture is no longer split between old and new semantic paths. +- Freestanding target foundations. +- Kernel capability proof before 1.0 stabilization. -- Standard library contracts for real programs (HTTP, filesystem/paths, process, env, time, logging, config) -- Capability-based access model for IO/process/env/network (secure-by-default for tools) -- Interactive execution engine: `incan run -i` (expression-first) → eventual Jupyter/kernel interop → richer workspace UX -- Packaging/distribution story for tools and projects (reproducible builds, artifact creation) -- Rust-hosted Incan caller boundary for native Rust applications consuming Incan-authored libraries ([RFC 097](RFCs/097_rust_hosted_incan_caller.md)) +## Release Milestones -## Status by Area (high-level) +### 0.4 Release: tooling and inspection -- Core language: see [RFC 000] / [RFC 008] -- Tooling (build/run/fmt/test): see the CLI docs and [RFC 001]/[RFC 002]/[RFC 004]/[RFC 007] for the planned testing surface -- Rust interop: see [RFC 005] / [RFC 013] and the [Rust Interop guide](language/how-to/rust_interop.md) -- Rust-hosted Incan consumption: see [RFC 097](RFCs/097_rust_hosted_incan_caller.md) for the proposed caller boundary between native Rust applications and Incan-authored libraries -- Web: see [Web Framework guide](language/tutorials/web_framework.md) (stabilization ongoing); interactive runtime stdlib contracts in [RFC 092](RFCs/092_interactive_runtime_stdlib_contracts.md) +The 0.4 milestone is the tooling and inspection release. It focuses on: -## Upcoming (next) +- canonical SDK install path; +- zero-clone starter flow; +- first-contact docs and positioning; +- stable machine-readable diagnostics; +- diagnostic explain catalog; +- codegraph export for agent/maintainer code intelligence; +- generated Rust and emitted artifact inspection; +- build reports. -- Interactive runtime stdlib contracts per [RFC 092](RFCs/092_interactive_runtime_stdlib_contracts.md) (target manifests, host capabilities, execution regions, artifact metadata, diagnostics, input/accessibility hooks, optional GPU surfaces) -- Test runner fixture execution (setup/teardown lifecycle) -- Dev server + prod build pipeline for WASM target -- Python-style generators ([RFC 006]) — `yield` + `Generator[T]` satisfying the iteration protocol -- Inline tests ([RFC 007]) — `@test` in source files, Rust-style proximity -- **Later / superseded by narrower RFCs:** WASM/JSX-native parser & codegen, `--target wasm`, dev server + prod pipeline tuned for WASM, WebGPU-style 3D, and broader non-browser/native-class runtime targets should advance only through focused RFCs after the stdlib contracts in RFC 092 are validated +New language/runtime feature work is out of scope unless it directly supports that tooling path. + +Core tracking issues: + +- [#223](https://github.com/dannys-code-corner/incan/issues/223): 0.4 tooling, inspection, and first-contact umbrella. +- [#428](https://github.com/dannys-code-corner/incan/issues/428): canonical SDK installer and release manifest. +- [#553](https://github.com/dannys-code-corner/incan/issues/553): zero-clone starter project flow. +- [#551](https://github.com/dannys-code-corner/incan/issues/551): first-contact quickstart and positioning docs. +- [#554](https://github.com/dannys-code-corner/incan/issues/554): release direction notes and scope guard. +- [#573](https://github.com/dannys-code-corner/incan/issues/573): codegraph export. +- [#589](https://github.com/dannys-code-corner/incan/issues/589): stable JSON diagnostics. +- [#590](https://github.com/dannys-code-corner/incan/issues/590): diagnostic explain catalog. +- [#591](https://github.com/dannys-code-corner/incan/issues/591): build artifact report. +- [#567](https://github.com/dannys-code-corner/incan/issues/567): generated Rust inspection tooling and quality gates. +- [#592](https://github.com/dannys-code-corner/incan/issues/592): RFC template inspectability prompts, if tiny and opportunistic. + +### 0.5 Release: backend foundation and Hees.ai proof lane + +The 0.5 milestone begins deprecating the Rust-source backend as the semantic path. It introduces the compiler foundations needed for a backend-neutral middle end: + +- stable compiler IDs; +- backend-neutral semantic facts; +- `IncanType` and semantic type modeling; +- ABI v0 design hooks; +- HIR v0; +- behavior inventory; +- backend migration scaffolding. + +Stdlib RFC/work is allowed in this lane. Hees.ai is also allowed, but only as a constrained commercial and dogfood proof path that validates compiler, stdlib, runtime, and tooling direction. Hees.ai work should consume general Incan surfaces, not quietly become broad product scope inside the language milestone. + +Core tracking issues: + +- [#634](https://github.com/dannys-code-corner/incan/issues/634): v1.0 middle-end foundation umbrella. +- [#646](https://github.com/dannys-code-corner/incan/issues/646): current compiler behavior inventory. +- [#647](https://github.com/dannys-code-corner/incan/issues/647): deprecate Rust-source backend as semantic path. +- [#648](https://github.com/dannys-code-corner/incan/issues/648): stable compiler IDs and semantic facts database. +- [#649](https://github.com/dannys-code-corner/incan/issues/649): `IncanType` semantic type model and ABI v0 hooks. +- [#650](https://github.com/dannys-code-corner/incan/issues/650): HIR v0 and snapshot tests. +- [#282](https://github.com/dannys-code-corner/incan/issues/282): backend orchestration migration scaffolding. +- [#224](https://github.com/dannys-code-corner/incan/issues/224): `CompilationSession` semantic database transition. +- [#549](https://github.com/dannys-code-corner/incan/issues/549): Hees.ai governed workbench demo. +- [#651](https://github.com/dannys-code-corner/incan/issues/651): Hees.ai dependency inventory and guardrails. + +Allowed stdlib work includes `std.http`, `std.ci`, CLI framework, `std.archive`, `std.process`, `std.web` lifecycle, `std.environ`, package-level timezones, fallible reader chunk streams, and selected stdlib compilation/source-authored behavior work. + +### 0.6 Release: backend cutover + +The 0.6 milestone removes the Rust-source backend from the normal compiler path. The replacement backend should preserve supported behavior, report compatibility/migration details, and retire generated Rust as the semantic handoff. + +Only runtime/DSL RFC scope that stress-tests or supports the new backend belongs here. + +Core tracking issues: + +- [#652](https://github.com/dannys-code-corner/incan/issues/652): replacement backend parity cutover. +- [#653](https://github.com/dannys-code-corner/incan/issues/653): Body IR v0 and backend-owned lowering. +- [#654](https://github.com/dannys-code-corner/incan/issues/654): remove Rust-source backend and generated-Rust semantic handoff. +- [#655](https://github.com/dannys-code-corner/incan/issues/655): backend compatibility report and migration notes. +- [#225](https://github.com/dannys-code-corner/incan/issues/225): semantic facts adoption on backend cutover paths. +- [#656](https://github.com/dannys-code-corner/incan/issues/656): Rust-facing ABI and Cargo-native Incan package direction. +- [RFC 092](RFCs/092_interactive_runtime_stdlib_contracts.md): interactive runtime stdlib contracts. +- [RFC 093](RFCs/093_std_telemetry_opentelemetry_observability.md): `std.telemetry`. +- [RFC 094](RFCs/094_context_managers.md): context managers. +- [RFC 095](RFCs/095_span_vocabulary_blocks.md): span vocabulary blocks. + +### 0.7 Release: feature reopening + +The 0.7 milestone is the broader feature reopening lane after the backend replacement is complete. This is where deferred language, package, registry, lifecycle, interop, docs-generation, editor, and product-surface work can resume. + +Examples of deferred lanes: + +- incan.pub and package registry/product identity. +- InQL and Pallay SDK dogfood. +- source-local feature metadata. +- Python interop research. +- checked API docs generation. +- Windows/package-manager/self-upgrade convenience work. +- trait/newtype language features not required by backend cutover. +- broader editor and package lifecycle work. + +0.7 should not absorb freestanding/kernel primitives by default. That work needs its own release lanes so feature reopening does not become the place where unsafe, layout, target, runtime, and kernel proof work all land at once. + +### 0.8 Release: freestanding foundations + +The 0.8 milestone defines the compiler, runtime, ABI, and package foundations needed for freestanding targets. It should make low-level targets possible without promising a production kernel or stabilizing every low-level surface. + +The release should answer how Incan code can compile without assuming hosted `std`, a process environment, filesystem access, threads, default allocator availability, or ordinary hosted panic behavior. + +Expected scope: + +- freestanding target profiles and capability manifests; +- runtime layering across `core`, `alloc`, hosted `std`, and future kernel-facing APIs; +- no-std/freestanding build mode; +- panic strategy and allocator hooks; +- ABI/layout/repr/alignment/calling-convention controls; +- an explicit unsafe model for raw pointers, volatile access, MMIO, and low-level intrinsics; +- package metadata for freestanding compatibility. + +Core tracking issues: + +- [#681](https://github.com/dannys-code-corner/incan/issues/681): RFC proposal for freestanding targets and runtime layering. +- [#682](https://github.com/dannys-code-corner/incan/issues/682): RFC proposal for unsafe blocks and low-level operations. +- [#683](https://github.com/dannys-code-corner/incan/issues/683): RFC proposal for representation, layout, and calling convention controls. +- [#684](https://github.com/dannys-code-corner/incan/issues/684): stdlib/runtime layer inventory for freestanding foundations. +- [#685](https://github.com/dannys-code-corner/incan/issues/685): freestanding target profiles and runtime requirement reports. +- [#686](https://github.com/dannys-code-corner/incan/issues/686): no-std freestanding build mode and restricted artifact smoke test. +- [#687](https://github.com/dannys-code-corner/incan/issues/687): unsafe low-level operation surface v0. +- [#688](https://github.com/dannys-code-corner/incan/issues/688): layout, repr, and calling-convention metadata v0. +- [#689](https://github.com/dannys-code-corner/incan/issues/689): panic strategy and allocator hooks for freestanding targets. + +0.8 is successful when Incan can compile a restricted freestanding artifact and report which runtime, allocator, panic, target, and ABI capabilities it requires. + +### 0.9 Release: kernel capability proof + +The 0.9 milestone is the vertical proof that the freestanding foundations work under real low-level pressure. It should boot a tiny Incan-authored kernel under an emulator, not ship a production operating system. + +Expected scope: + +- minimal architecture support layer; +- linker and boot configuration; +- QEMU runner and smoke harness; +- serial output; +- panic halt/report path; +- allocator hookup; +- MMIO/volatile/raw pointer use; +- one interrupt, timer, or simple task proof. + +Core tracking issues: + +- [#690](https://github.com/dannys-code-corner/incan/issues/690): QEMU tiny kernel capability proof. + +0.9 is successful when Incan can build and boot a tiny freestanding kernel under QEMU with Incan-authored init logic and a concrete low-level capability proof. + +### 1.0 Release: stabilization and public contracts + +The 1.0 milestone consolidates the post-cutover compiler architecture, ABI/package direction, tooling contracts, stdlib maturity, ecosystem workflows, freestanding lessons, and documentation into a coherent public surface. + +1.0 should describe what Incan is, what it guarantees, how packages and generated artifacts are consumed, where Rust-facing interop boundaries are stable, and which freestanding/kernel-facing surfaces are stable, experimental, or intentionally deferred. + +## Status by Area + +- Core language: see [RFC 000] / [RFC 008]. +- Testing surface: see [RFC 018] / [RFC 019] / [RFC 004]. +- Tooling and first-contact: install, starter, diagnostics, explain, codegraph, artifact inspection, and build reports are the immediate release surface. +- Rust interop: see [RFC 005] / [RFC 013] and the [Rust Interop guide](language/how-to/rust_interop.md). Rust-hosted consumption should be reframed through ABI and Cargo-native package direction instead of generated Rust as the public semantic path. +- Web and interactive runtime: see the [Web Framework guide](language/tutorials/web_framework.md), [RFC 092](RFCs/092_interactive_runtime_stdlib_contracts.md), and related runtime/DSL RFCs. +- Standard library: stdlib work is allowed in the backend-foundation lane where it helps real programs and dogfood paths validate compiler/runtime direction. ## Deferred / Later -The following items are intentionally deferred to later, and might be revisited in the future: +The following items remain intentionally deferred until they have a focused RFC or implementation lane: -- SSR/SSG for frontend: Server-Side Rendering / Static Site Generation for the WASM/UI stack (render pages ahead of time - or on the server, then hydrate). -- Desktop/mobile via wgpu: using the wgpu graphics stack to run Incan apps as native desktop/mobile apps (instead of - browser-only). -- CRDT/collab features: real-time collaboration primitives (Conflict-free Replicated Data Types) for things like - collaborative editing, shared state, etc. +- SSR/SSG for frontend: Server-Side Rendering / Static Site Generation for the WASM/UI stack (render pages ahead of time or on the server, then hydrate). +- Desktop/mobile via wgpu: using the wgpu graphics stack to run Incan apps as native desktop/mobile apps instead of browser-only. +- CRDT/collab features: real-time collaboration primitives (Conflict-free Replicated Data Types) for collaborative editing, shared state, and similar workflows. ### Guides diff --git a/workspaces/docs-site/docs/tooling/reference/checked_api_metadata.md b/workspaces/docs-site/docs/tooling/reference/checked_api_metadata.md index 2bb9e4d46..a6f83025b 100644 --- a/workspaces/docs-site/docs/tooling/reference/checked_api_metadata.md +++ b/workspaces/docs-site/docs/tooling/reference/checked_api_metadata.md @@ -163,12 +163,16 @@ The metadata is derived from parsed and typechecked semantics. Public declaratio - public partial callable presets with target provenance, preset metadata, projected callable parameters, return type, and async status - raw docstring text when the declaration or method has a docstring - parsed docstring sections in `docstring_sections`, including summary, parameters, returns, fields, aliases, and decorators -- decorator metadata with resolved decorator paths +- decorator metadata with resolved decorator paths, safe argument projections, and decorated callable context when the decorator is attached to a callable declaration - safe const values for public consts and safe decorator arguments Types use the same structural `TypeRef` encoding as library manifest exports. For example, a non-generic type is encoded as `{"Named": {"name": "str"}}`, while a generic application is encoded as `{"Applied": {"name": "List", "args": [...]}}`. -When decorator processing exposes a public function as a callable-valued binding, metadata follows that checked binding. In that case, function metadata reports the callable binding's parameters and return type rather than the original source signature. Existing decorator metadata remains attached separately through `decorators`, so consumers that inspect marker decorators, safe decorator arguments, or docstring `Decorators:` sections can keep using that lane without inferring binding types from it. +Function metadata keeps the source declaration's public callable surface. For a decorated callable, each decorator entry also carries `decorated_callable`, which contains the decorated declaration's checked public identity, source anchor, type parameters, parameter names and types, return type, receiver when applicable, and async marker. Registry and catalog tooling should read that field instead of asking authors to repeat the decorated function name or signature in decorator arguments. + +Decorator arguments are represented structurally when the compiler can do so without executing user code. Literals, checked const references, symbolic references, lists, dicts, constructors, and ordinary calls can appear as metadata values. Unsupported expressions remain explicit `unsupported` entries. + +Public import aliases can include `projected_function` when the alias target resolves to a public function or callable-valued decorated binding. The projection includes the source declaration path, the callable signature under the alias name, and the source decorators. This lets reexport-only facades expose declaration metadata without no-op loader functions or runtime module initialization hooks. Public partial declarations use `kind: "partial"`. A partial declaration remains distinct from a hand-written function or alias: @@ -198,7 +202,7 @@ Metadata only carries values that the compiler can expose without executing user | `bytes` | Bytes literal or frozen bytes const | | `none` | Literal `None` | -Decorator arguments that are not literals, type arguments, or const references are reported as `unsupported` metadata values instead of being evaluated. +Decorator arguments that are not declaration-safe literals, const references, symbolic references, lists, dicts, constructors, or ordinary call trees are reported as `unsupported` metadata values instead of being evaluated. ## Docstrings @@ -242,4 +246,4 @@ The metadata JSON describes public declarations from checked Incan source and ma Checked API metadata extraction does not inspect built `.incnlib` artifacts. Artifact inspection remains a separate tooling surface from source/project metadata extraction. -The extractor exposes only checked compiler facts and safe literal/const values. Unsupported decorator expressions are reported as `unsupported` metadata rather than evaluated, and consumers should not treat docstrings or decorator payloads as trusted executable input. +The extractor exposes only checked compiler facts and declaration-safe metadata values. Unsupported decorator expressions are reported as `unsupported` metadata rather than evaluated, and consumers should not treat docstrings or decorator payloads as trusted executable input. diff --git a/workspaces/docs-site/requirements-docs.txt b/workspaces/docs-site/requirements-docs.txt index c1caebdd8..e77730665 100644 --- a/workspaces/docs-site/requirements-docs.txt +++ b/workspaces/docs-site/requirements-docs.txt @@ -3,6 +3,6 @@ mkdocs-material==9.5.49 mike==2.1.3 mkdocs-redirects==1.2.1 mkdocs-gen-files==0.6.0 -pymdown-extensions==10.21.2 -# PyMdown 10.21.2 handles Pygments 2.20+ when no fence title produces filename=None. +pymdown-extensions==10.21.3 +# PyMdown 10.21.3 handles Pygments 2.20+ when no fence title produces filename=None. pygments==2.20.0