diff --git a/.planning/http-servers-standalone-binaries.plan.md b/.planning/http-servers-standalone-binaries.plan.md new file mode 100644 index 0000000..b43f02a --- /dev/null +++ b/.planning/http-servers-standalone-binaries.plan.md @@ -0,0 +1,204 @@ +# HTTP Servers as Standalone Binaries Plan + +## Roadmap Item + +- `ROADMAP.md`: `HTTP servers work as standalone binaries` + +## Production-Readiness Status + +Forge is now serious beta for language/runtime development: CI covers Rust tests on Linux/macOS/Windows, Forge integration tests, backend parity, audit, OTel build, and a real `fork_for_serving` benchmark. The server runtime is no longer blocked by global interpreter serialization. + +This item closes a visible production gap: a Forge HTTP service should be buildable into a single executable that does not shell out to the `forge` CLI at runtime. This is a **source-runtime standalone binary**: it statically links the Forge runtime and interpreter, embeds source, and boots the existing server runtime in-process. It does not make handlers true native-code functions yet; startup and 2-5x Rust performance are separate roadmap items. + +## Current State + +- `src/native.rs::build_native_aot` can build a standalone executable only when `libforge_lang.a` is discoverable. That executable embeds bytecode and calls `forge_execute_bytecode`. +- `src/lib.rs::forge_execute_bytecode` deserializes bytecode and runs the VM. It does not know how to launch decorator-driven HTTP servers. +- `src/main.rs::compile_to_native_aot` rejects programs with decorators because `ensure_vm_compatible` marks `@server` / `@get` runtime metadata as VM-incompatible. This should stay true: decorated servers are not bytecode AOT today. +- `src/main.rs::compile_to_native_launcher` accepts source programs, but the generated binary writes a temp `.fg` and execs the `forge` CLI. That is not standalone. +- `src/runtime/host.rs::launch` already knows how to spawn schedules/watchers and start HTTP servers from a parsed program plus an initialized `Interpreter`. + +## Scope + +### In Scope + +1. Add a standalone source-execution FFI entrypoint in `src/lib.rs` for embedded source programs. +2. Teach `src/native.rs` to emit a C wrapper that embeds Forge source and calls the new source-execution entrypoint when `libforge_lang.a` is available. +3. Route `forge build --native` decorated server programs through the standalone source path instead of a CLI-shellout launcher when `libforge_lang.a` is available. +4. Add focused tests proving: + - the source FFI path can run non-server source, + - a decorated server can be built into a standalone executable when `libforge_lang.a` is available, + - launcher fallback behavior remains available when no static library is discoverable. + +### Out of Scope + +- True native route function pointers via `forge_register_route`. +- Cranelift AOT codegen for decorated handlers. +- `forge build --native --aot` support for decorated servers. Keep `--aot` VM-bytecode-only and direct users to `--native` for standalone source-runtime servers. +- Startup-time target `< 10ms`. +- Performance target `2-5x of equivalent Rust`. +- Cross-compilation target support. +- Windows standalone static linking. Existing standalone AOT is Unix-only; keep that boundary unless implementation proves a tiny safe Windows slice is available. + +## Approach + +### U1. Share the Source Runtime Pipeline + +Files: +- `src/lib.rs` +- `src/runtime/embedded.rs` (new) or another small runtime helper module + +Extract a shared Rust helper for the same two-phase flow used by `forge run`: + +Behavior: +- Parse source with existing `Lexer` / `Parser`. +- Create an `Interpreter`, set `source`, set `source_file` or diagnostic source label, and set `defer_host_runtime = true`. +- Optionally call `permissions::set_allow_run(allow_run)`. +- Run the interpreter once so top-level bindings and functions are installed. +- Extract `RuntimePlan` and call `runtime::host::launch`. +- Create and own a Tokio multi-thread runtime (`Builder::new_multi_thread().enable_all()`) for the embedded entrypoint before calling `host::launch`. +- Enter that runtime with `rt.block_on(async { ... host::launch(...).await ... })`; merely constructing a `Runtime` is not enough because stdlib modules use `tokio::runtime::Handle::try_current()`. +- Let `start_server` keep ownership of OTel/subscriber ordering; it already calls `init_otel()` from inside the Tokio runtime before `init_subscriber()`. +- For non-server source, return after the top-level interpreter run and empty runtime launch. +- Preserve current `forge run` behavior for schedule/watch-only programs: without a server, launch returns after spawning background threads and the process exits when the main thread returns. + +Rationale: +- Keeps `main.rs` and embedded/native source execution from drifting. +- Makes schedules/watchers/server launch behavior match `forge run`. + +Tests: +- Unit-test simple non-server source exits successfully through the helper. +- Unit-test `@server` without routes returns the existing runtime error. +- Unit-test shell builtins remain denied unless `allow_run` is true. +- Unit-test that both `Interpreter::source` and `Interpreter::source_file` / diagnostic label are populated for embedded source execution. + +### U2. Add Source Execution FFI + +Files: +- `src/lib.rs` + +Add `forge_execute_source(source_ptr, source_len, path_ptr, path_len, allow_run) -> i32` or a similarly explicit options-shaped C ABI. + +Behavior: +- Validate non-null source pointer and nonzero length. +- Decode UTF-8 source and optional path. +- Call the shared source runtime helper from U1. +- Wrap the call in `panic::catch_unwind(AssertUnwindSafe(...))` so panics never unwind across C. +- Return `0` on success, `1` on user/runtime/frontend errors, and `1` with a stable stderr message on panic. + +Safety contract: +- Document that pointer+length pairs must reference valid memory for the duration of the call. +- The C wrapper generated by Forge is the primary caller; arbitrary embedders get a best-effort status code, not a reusable-process guarantee after panic. +- If the Rust signature is `pub unsafe extern "C" fn`, generated C remains unchanged, but Rust callers must acknowledge the raw-pointer contract. +- `allow_run` writes to process-global permission state. This is acceptable for generated standalone binaries, which call the entrypoint once per process, but arbitrary multi-call embedders should not rely on per-call isolation. + +Security decision: +- Do not enable shell execution implicitly. +- Thread the existing CLI `--allow-run` flag into `forge build --native` so `forge build --native --allow-run app.fg` bakes `allow_run = true` into the generated wrapper. +- Declare the build subcommand's `allow_run` option as native-only (`requires = "native"` or `conflicts_with = "aot"`) so `forge build --aot --allow-run` is rejected as meaningless. +- Default standalone binaries keep shell builtins denied, matching file execution security. + +Tests: +- Unit-test invalid/null input returns failure. +- Unit-test invalid UTF-8 returns failure. +- Unit-test `allow_run = false` rejects a shell builtin and `allow_run = true` permits it. + +### U3. Emit Standalone Source Wrappers + +Files: +- `src/native.rs` + +Add a standalone source builder alongside `build_standalone_aot`: +- Embed source bytes as `FORGE_SOURCE`. +- Embed a diagnostic source label, preferably the source file basename or caller-provided display name rather than an absolute build-machine path. +- Link against `libforge_lang.a`. +- Call `forge_execute_source(FORGE_SOURCE, FORGE_SOURCE_LEN, FORGE_SOURCE_PATH, FORGE_SOURCE_PATH_LEN, FORGE_ALLOW_RUN)`. + +Change `build_native_launcher`: +- If `find_libforge_dir()` succeeds, build a standalone source-runtime binary. +- If not, preserve current launcher behavior that shells out to `forge`. +- Make this behavior explicit in CLI output so users know whether the produced binary is standalone or a CLI launcher. + +Tests: +- C source generation contains `forge_execute_source` and embeds source bytes, not a temp-file exec path. +- Existing launcher generation tests still cover fallback C source. +- `build_native_launcher` standalone smoke test is gated on Unix + `cc` + discoverable `libforge_lang.a`. +- Tests assert the generated C does not embed absolute source paths by default. + +### U4. Keep AOT Honest and Route Native Builds + +Files: +- `src/main.rs` +- `src/native.rs` + +Behavior: +- `forge build --native app.fg` builds a standalone source-runtime binary when `libforge_lang.a` is available; otherwise it builds the existing CLI launcher. +- `forge build --native --allow-run app.fg` bakes shell permission into the generated standalone source wrapper. +- `forge build --aot --allow-run app.fg` is rejected at CLI parsing or validation. +- `forge build --aot app.fg` remains bytecode/VM-only. +- `forge build --aot server.fg` still rejects decorators, but the error should clearly say: `decorator-driven servers are not bytecode AOT yet; use forge build --native for a standalone source-runtime server binary`. + +Rationale: +- This satisfies the roadmap item without pretending decorated handlers have native codegen. +- It preserves current bytecode AOT behavior for VM-compatible programs. + +Tests: +- CLI-level unit or integration test for `--native` decorator program selecting source-standalone path when the static library is available. +- CLI-level test for `--aot` decorator program rejecting with the new honest guidance. + +### U5. End-to-End Native Server Smoke + +Files: +- `tests/native_server.rs` or existing native test module +- Optional fixture under `tests/fixtures/` or `examples/` + +Add a Unix-gated smoke test: +- Build `libforge_lang.a` if needed or skip with a clear message if unavailable. +- Build a tiny server program with `@server(port: )` and one `@get("/ping")`. +- Start the produced binary as a child process. +- Poll `/ping` until success. +- Send SIGTERM when available, wait briefly for graceful shutdown, then kill as cleanup fallback. +- Capture stderr and assert the server reaches normal startup logging; this protects the embedded Tokio/OTel initialization path from panicking before serving. + +Guardrails: +- Use an ephemeral port inserted into the source before build. +- Time out aggressively so CI cannot hang. +- Skip if no `cc` or static library is unavailable in the test environment. +- Record binary size in test output and keep a loose upper ceiling if practical, because source-runtime binaries link the full Forge runtime. + +## Edge Cases + +- Invalid UTF-8 source passed through FFI returns failure. +- Source path may be omitted by C caller; diagnostics should still work. +- Server programs run forever; the native smoke test must always kill the child. +- Shell builtins remain denied by default in standalone binaries. +- `--allow-run` can be baked into a standalone binary only when explicitly supplied at build time. +- `ALLOW_RUN` remains process-global; standalone generated binaries are one-call processes, but embedding multiple Forge executions with different permissions in one host process is not supported by this slice. +- `@server` without routes should return the existing runtime error. +- If `libforge_lang.a` is absent, existing launcher fallback remains unchanged. +- Source is embedded in plaintext in the binary. This is acceptable for this roadmap slice and must be documented in CLI output or docs; bytecode/source-hiding remains the bytecode AOT path for VM-compatible programs. + +## Rollback Plan + +- Remove `forge_execute_source`. +- Remove standalone source wrapper generation. +- Remove the improved `--aot` guidance and keep decorator rejection unchanged. +- Remove native server smoke tests. + +## Verification + +- `cargo fmt -- --check` +- `cargo test` +- `cargo clippy --all-targets -- -A clippy::approx_constant -A clippy::result_large_err -A clippy::only_used_in_recursion -A clippy::len_zero` +- `cargo run -- --allow-run test tests/` +- `cargo build` +- Targeted native/server smoke tests added by this plan +- Update `CHANGELOG.md` under `[Unreleased]` because `forge build --native` behavior changes for users with `libforge_lang.a` available. + +## Success Criteria + +- A decorated Forge HTTP server can be built into a single executable that does not call the `forge` CLI at runtime when linked with `libforge_lang.a`. +- Existing launcher fallback still works when no static library is available. +- Existing bytecode AOT behavior for VM-compatible programs remains unchanged. +- `--aot` remains honest: decorated servers are rejected with guidance to use `--native`. +- Tests and CLI output make the source-runtime boundary explicit so future true AOT work can replace it deliberately. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ef9050..e4ee7d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Standalone source-runtime native binaries for Forge servers** — `forge build --native` now links against `libforge_lang.a` when available and emits a single executable that embeds Forge source and starts interpreter-only runtime features like `@server` without shelling out to the `forge` CLI. `--aot` remains bytecode/VM-only and continues to reject decorator-driven servers with guidance to use `--native`. - **Structured concurrency with `squad` blocks** — `squad { spawn { } spawn { } }` runs tasks concurrently with automatic join, cooperative cancellation on failure, and error propagation. Returns an array of results in spawn order. Works in both interpreter and VM engines. - **First-class `Set` type** — `set([1, 2, 3])` or `set((1, 2, 3))` builds a deduplicated set. Methods: `.has(x)`, `.add(x)`, `.remove(x)`, `.union(other)`, `.intersect(other)`, `.diff(other)`, `.to_array()`. Supports `len()`, `contains()`, iteration, order-independent equality, and is truthy when non-empty. Works across interpreter, VM, bytecode round-trip, and JIT. - **First-class `Map` type** — `map([("a", 1), ("b", 2)])` or `map()` builds an ordered key/value map with any-type keys. Methods: `.get(k)`, `.set(k, v)`, `.has(k)`, `.remove(k)`, `.keys()`, `.values()`, `.len()`, `.to_array()`. Insertion order is preserved on overwrite. Key equality uses container semantics (int/float collision, NaN self-match). Supports `for k, v in m` iteration (which also unlocks `for k, v in obj` parity for plain objects under the VM), `len()`, `contains()`, order-independent equality, and is truthy when non-empty. `json.stringify` emits JSON objects for maps with string keys and errors on non-string keys. Works across interpreter, VM, bytecode round-trip, and JIT. diff --git a/src/lib.rs b/src/lib.rs index bd34629..62fb426 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -16,7 +16,7 @@ mod stdlib; mod typechecker; pub mod vm; -use std::panic; +use std::panic::{self, AssertUnwindSafe}; /// Execute serialized bytecode. Returns 0 on success, 1 on error. /// @@ -61,3 +61,72 @@ pub extern "C" fn forge_execute_bytecode(bytecode_ptr: *const u8, bytecode_len: } } } + +/// Execute embedded Forge source. Returns 0 on success, 1 on error. +/// +/// This is the source-runtime standalone entrypoint used by generated native +/// wrappers for programs that need interpreter-only features such as +/// decorator-driven HTTP servers. +/// +/// # Safety +/// `source_ptr` must point to `source_len` valid bytes of UTF-8 Forge source +/// for the duration of this call. When `path_len > 0`, `path_ptr` must point to +/// `path_len` valid bytes of UTF-8 diagnostic label data. +#[no_mangle] +pub unsafe extern "C" fn forge_execute_source( + source_ptr: *const u8, + source_len: usize, + path_ptr: *const u8, + path_len: usize, + allow_run: i32, +) -> i32 { + if source_ptr.is_null() || source_len == 0 { + eprintln!("forge: null or empty source"); + return 1; + } + if path_ptr.is_null() && path_len > 0 { + eprintln!("forge: null source path with nonzero length"); + return 1; + } + + let result = panic::catch_unwind(AssertUnwindSafe(|| { + let source_bytes = unsafe { std::slice::from_raw_parts(source_ptr, source_len) }; + let source = match std::str::from_utf8(source_bytes) { + Ok(source) => source, + Err(err) => { + eprintln!("forge: source is not valid UTF-8: {err}"); + return 1; + } + }; + + let source_label = if path_len == 0 { + "".to_string() + } else { + let path_bytes = unsafe { std::slice::from_raw_parts(path_ptr, path_len) }; + match std::str::from_utf8(path_bytes) { + Ok(path) => path.to_string(), + Err(err) => { + eprintln!("forge: source path is not valid UTF-8: {err}"); + return 1; + } + } + }; + + let config = runtime::embedded::EmbeddedSourceConfig::new(source_label, allow_run != 0); + match runtime::embedded::execute_source_standalone(source, config) { + Ok(()) => 0, + Err(err) => { + eprintln!("{err}"); + 1 + } + } + })); + + match result { + Ok(code) => code, + Err(_) => { + eprintln!("forge: internal panic during source execution"); + 1 + } + } +} diff --git a/src/main.rs b/src/main.rs index 42ab4a8..6b1409c 100644 --- a/src/main.rs +++ b/src/main.rs @@ -148,6 +148,9 @@ enum Command { /// Compile to bytecode and embed in a native binary (no source exposure) #[arg(long, conflicts_with = "native")] aot: bool, + /// Bake shell execution permission into a --native standalone source-runtime binary + #[arg(long = "allow-run", requires = "native", conflicts_with = "aot")] + allow_run: bool, /// Source file to compile file: PathBuf, }, @@ -318,7 +321,12 @@ async fn main() { Some(Command::New { name }) => { scaffold::create_project(&name); } - Some(Command::Build { file, native, aot }) => { + Some(Command::Build { + file, + native, + aot, + allow_run: build_allow_run, + }) => { let path_str = file.display().to_string(); let source = match fs::read_to_string(&file) { Ok(s) => s, @@ -336,7 +344,13 @@ async fn main() { if aot { compile_to_native_aot(&source, &path_str, &file, strict); } else if native { - compile_to_native_launcher(&source, &path_str, &file, strict); + compile_to_native_launcher( + &source, + &path_str, + &file, + strict, + cli.allow_run || build_allow_run, + ); } else { compile_to_bytecode(&source, &path_str, &file, strict); } @@ -979,19 +993,34 @@ fn compile_to_bytecode(source: &str, filename: &str, file_path: &PathBuf, strict } } -fn compile_to_native_launcher(source: &str, filename: &str, file_path: &PathBuf, strict: bool) { +fn compile_to_native_launcher( + source: &str, + filename: &str, + file_path: &PathBuf, + strict: bool, + allow_run: bool, +) { let (_, warnings) = match prepare_program(source, strict) { Ok(prepared) => prepared, Err(err) => print_frontend_error(source, filename, err), }; emit_type_warnings(&warnings); - match native::build_native_launcher(source, file_path) { - Ok(output_path) => { + match native::build_native_launcher(source, file_path, allow_run) { + Ok(output) => { + let runtime_msg = match output.runtime { + native::NativeRuntimeKind::StandaloneSourceRuntime => { + "standalone source runtime (libforge linked; source embedded)" + } + native::NativeRuntimeKind::CliLauncher => { + "Forge CLI launcher required at execution time" + } + }; println!( - "Built native launcher {} -> {}\n runtime: Forge interpreter/VM required at execution time", + "Built native binary {} -> {}\n runtime: {}", filename, - output_path.display() + output.path.display(), + runtime_msg, ); } Err(message) => { @@ -1009,6 +1038,13 @@ fn compile_to_native_aot(source: &str, filename: &str, file_path: &PathBuf, stri emit_type_warnings(&warnings); if let Err(message) = ensure_vm_compatible(&program, "AOT build") { + let message = if message.contains("decorator-driven runtime features") { + format!( + "{message}\n hint: decorator-driven servers are not bytecode AOT yet; use `forge build --native` for a standalone source-runtime server binary" + ) + } else { + message + }; eprintln!("{}", errors::format_simple_error(&message)); process::exit(1); } @@ -1115,6 +1151,14 @@ mod tests { assert!(help.contains("falls back to the bytecode interpreter automatically")); } + #[test] + fn build_allow_run_is_native_only() { + assert!( + Cli::try_parse_from(["forge", "build", "--native", "--allow-run", "app.fg"]).is_ok() + ); + assert!(Cli::try_parse_from(["forge", "build", "--aot", "--allow-run", "app.fg"]).is_err()); + } + #[test] fn parity_corpus_supported_cases() { let cases = crate::testing::parity::load_supported_cases(); diff --git a/src/native.rs b/src/native.rs index 79e5e5d..d489163 100644 --- a/src/native.rs +++ b/src/native.rs @@ -4,9 +4,18 @@ use std::path::{Path, PathBuf}; use std::process::Command; use std::time::{SystemTime, UNIX_EPOCH}; -pub fn build_native_launcher(source: &str, source_path: &Path) -> Result { +pub fn build_native_launcher( + source: &str, + source_path: &Path, + allow_run: bool, +) -> Result { + if let Some(lib_dir) = find_libforge_dir() { + return build_standalone_source(source, source_path, allow_run, &lib_dir) + .map(NativeBuildOutput::standalone); + } + let c_source_fn = |forge_bin: &str| native_launcher_c_source(source.as_bytes(), forge_bin); - compile_launcher(source_path, "native", c_source_fn) + compile_launcher(source_path, "native", c_source_fn).map(NativeBuildOutput::launcher) } pub fn build_native_aot(bytecode: &[u8], source_path: &Path) -> Result { @@ -19,6 +28,34 @@ pub fn build_native_aot(bytecode: &[u8], source_path: &Path) -> Result Self { + Self { + path, + runtime: NativeRuntimeKind::StandaloneSourceRuntime, + } + } + + fn launcher(path: PathBuf) -> Self { + Self { + path, + runtime: NativeRuntimeKind::CliLauncher, + } + } +} + /// Find the directory containing libforge_lang.a pub fn find_libforge_dir() -> Option { // Check FORGE_LIB_DIR env var first @@ -40,6 +77,80 @@ pub fn find_libforge_dir() -> Option { None } +/// Build a standalone source-runtime binary that links against libforge.a. +#[cfg(unix)] +fn build_standalone_source( + source: &str, + source_path: &Path, + allow_run: bool, + lib_dir: &Path, +) -> Result { + let output_path = native_output_path(source_path); + let c_source = standalone_source_c_source(source.as_bytes(), source_path, allow_run); + + let build_id = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map_err(|e| format!("failed to create timestamp: {e}"))? + .as_nanos(); + let c_path = env::temp_dir().join(format!( + "forge-source-standalone-{}-{}.c", + std::process::id(), + build_id + )); + fs::write(&c_path, c_source) + .map_err(|e| format!("failed to write standalone source wrapper: {e}"))?; + + let mut cmd = Command::new("cc"); + cmd.arg("-O2") + .arg(&c_path) + .arg("-o") + .arg(&output_path) + .arg(format!("-L{}", lib_dir.display())) + .arg("-lforge_lang") + .arg("-lm") + .arg("-lpthread") + .arg("-lresolv"); + + #[cfg(target_os = "macos")] + { + cmd.arg("-framework").arg("CoreFoundation"); + cmd.arg("-framework").arg("Security"); + cmd.arg("-framework").arg("SystemConfiguration"); + cmd.arg("-framework").arg("IOKit"); + cmd.arg("-liconv"); + } + + #[cfg(target_os = "linux")] + { + cmd.arg("-ldl"); + } + + let status = cmd.status().map_err(|e| { + let _ = fs::remove_file(&c_path); + format!("failed to invoke C compiler for standalone source runtime: {e}") + })?; + let _ = fs::remove_file(&c_path); + + if !status.success() { + return Err(format!( + "standalone source-runtime compilation failed for '{}'", + output_path.display() + )); + } + + Ok(output_path) +} + +#[cfg(not(unix))] +fn build_standalone_source( + _source: &str, + _source_path: &Path, + _allow_run: bool, + _lib_dir: &Path, +) -> Result { + Err("standalone source runtime is currently supported on Unix-like systems only".to_string()) +} + /// Build a standalone AOT binary that links against libforge.a #[cfg(unix)] fn build_standalone_aot( @@ -367,6 +478,58 @@ fn aot_launcher_c_source(bytecode: &[u8], default_forge_bin: &str) -> String { ) } +fn standalone_source_c_source(source: &[u8], source_path: &Path, allow_run: bool) -> String { + let source_bytes = source + .iter() + .map(|byte| byte.to_string()) + .collect::>() + .join(", "); + let label = source_path + .file_name() + .and_then(|name| name.to_str()) + .unwrap_or(""); + let label_bytes = label + .as_bytes() + .iter() + .map(|byte| byte.to_string()) + .collect::>() + .join(", "); + let allow_run = if allow_run { 1 } else { 0 }; + + format!( + r#"#include +#include + +extern int32_t forge_execute_source( + const uint8_t *source, + size_t source_len, + const uint8_t *path, + size_t path_len, + int32_t allow_run +); + +static const unsigned char FORGE_SOURCE[] = {{ {source_bytes} }}; +static const size_t FORGE_SOURCE_LEN = sizeof(FORGE_SOURCE); +static const unsigned char FORGE_SOURCE_PATH[] = {{ {label_bytes} }}; +static const size_t FORGE_SOURCE_PATH_LEN = sizeof(FORGE_SOURCE_PATH); +static const int32_t FORGE_ALLOW_RUN = {allow_run}; + +int main(void) {{ + return (int)forge_execute_source( + FORGE_SOURCE, + FORGE_SOURCE_LEN, + FORGE_SOURCE_PATH, + FORGE_SOURCE_PATH_LEN, + FORGE_ALLOW_RUN + ); +}} +"#, + source_bytes = source_bytes, + label_bytes = label_bytes, + allow_run = allow_run, + ) +} + fn c_string_escape(value: &str) -> String { value .chars() @@ -422,7 +585,8 @@ mod tests { let source_path = temp_root.join("hello.fg"); std::fs::write(&source_path, "println(\"hi\")").unwrap(); - let output_path = build_native_launcher("println(\"hi\")", &source_path).unwrap(); + let output = build_native_launcher("println(\"hi\")", &source_path, false).unwrap(); + let output_path = output.path; assert!(output_path.exists()); let metadata = std::fs::metadata(&output_path).unwrap(); #[cfg(unix)] @@ -446,6 +610,22 @@ mod tests { assert!(!c_source.contains("FORGE_PROGRAM[]")); } + #[test] + fn standalone_source_wrapper_calls_embedded_source_entrypoint() { + let source_path = Path::new("/private/build/app.fg"); + let c_source = standalone_source_c_source( + b"@server(port: 8080)\n@get(\"/ping\") fn ping() { return { ok: true } }", + source_path, + true, + ); + + assert!(c_source.contains("forge_execute_source")); + assert!(c_source.contains("static const unsigned char FORGE_SOURCE[]")); + assert!(c_source.contains("static const int32_t FORGE_ALLOW_RUN = 1")); + assert!(c_source.contains("FORGE_SOURCE_PATH")); + assert!(!c_source.contains("/private/build/app.fg")); + } + #[cfg(unix)] #[test] fn build_native_aot_emits_binary() { @@ -477,4 +657,81 @@ mod tests { let _ = std::fs::remove_file(output_path); let _ = std::fs::remove_dir_all(&temp_root); } + + #[cfg(unix)] + #[test] + fn standalone_source_server_binary_serves_ping_when_lib_available() { + if Command::new("cc").arg("--version").output().is_err() { + return; + } + + let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let lib_dir = manifest_dir.join("target").join("debug"); + if !lib_dir.join("libforge_lang.a").exists() { + eprintln!( + "skipping standalone server smoke: {} not found", + lib_dir.join("libforge_lang.a").display() + ); + return; + } + + let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind ephemeral port"); + let port = listener.local_addr().unwrap().port(); + drop(listener); + + let temp_root = std::env::temp_dir().join(format!( + "forge-native-server-test-{}", + SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir_all(&temp_root).unwrap(); + let source_path = temp_root.join("server.fg"); + let source = format!( + r#" + @server(port: {port}) + + @get("/ping") + fn ping() -> Json {{ + return {{ ok: true }} + }} + "# + ); + std::fs::write(&source_path, &source).unwrap(); + + let output_path = + build_standalone_source(&source, &source_path, false, &lib_dir).expect("build server"); + let mut child = Command::new(&output_path) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .spawn() + .expect("start standalone server"); + + let client = reqwest::blocking::Client::builder() + .timeout(std::time::Duration::from_millis(500)) + .build() + .expect("client"); + let url = format!("http://127.0.0.1:{port}/ping"); + let mut served = false; + for _ in 0..50 { + if client + .get(&url) + .send() + .map(|response| response.status().is_success()) + .unwrap_or(false) + { + served = true; + break; + } + std::thread::sleep(std::time::Duration::from_millis(100)); + } + + let _ = child.kill(); + let _ = child.wait(); + let _ = std::fs::remove_file(&output_path); + let _ = std::fs::remove_dir_all(&temp_root); + + assert!(served, "standalone server did not serve /ping on {url}"); + } } diff --git a/src/runtime/embedded.rs b/src/runtime/embedded.rs new file mode 100644 index 0000000..603b079 --- /dev/null +++ b/src/runtime/embedded.rs @@ -0,0 +1,126 @@ +use crate::interpreter::Interpreter; +use crate::lexer::Lexer; +use crate::parser::Parser; +use crate::permissions; +use std::path::{Path, PathBuf}; + +#[derive(Debug, Clone)] +pub struct EmbeddedSourceConfig { + pub source_label: String, + pub allow_run: bool, +} + +impl EmbeddedSourceConfig { + pub fn new(source_label: impl Into, allow_run: bool) -> Self { + Self { + source_label: source_label.into(), + allow_run, + } + } +} + +pub fn execute_source_standalone(source: &str, config: EmbeddedSourceConfig) -> Result<(), String> { + let runtime = tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .map_err(|err| format!("failed to create Tokio runtime: {err}"))?; + + runtime.block_on(execute_source_on_current_runtime(source, config)) +} + +pub async fn execute_source_on_current_runtime( + source: &str, + config: EmbeddedSourceConfig, +) -> Result<(), String> { + permissions::set_allow_run(config.allow_run); + + let mut lexer = Lexer::new(source); + let tokens = lexer + .tokenize() + .map_err(|err| format!("{}: {}", config.source_label, err))?; + let mut parser = Parser::new(tokens); + let program = parser + .parse_program() + .map_err(|err| format!("{}: {}", config.source_label, err))?; + + let mut interpreter = Interpreter::new(); + interpreter.source = Some(source.to_string()); + interpreter.source_file = source_file_label(&config.source_label); + interpreter.set_defer_host_runtime(true); + + interpreter + .run(&program) + .map_err(|err| format_runtime_error(source, &config.source_label, &err))?; + + let runtime_plan = super::metadata::extract_runtime_plan(&program); + super::host::launch(interpreter, &runtime_plan) + .await + .map_err(|err| err.message) +} + +fn source_file_label(label: &str) -> Option { + if label.is_empty() { + return None; + } + + Some(Path::new(label).to_path_buf()) +} + +fn format_runtime_error( + source: &str, + label: &str, + err: &crate::interpreter::RuntimeError, +) -> String { + if err.line > 0 { + crate::errors::format_error( + source, + err.line, + if err.col > 0 { err.col } else { 1 }, + &format!("[{}] {}", label, err.message), + ) + } else { + crate::errors::format_simple_error(&format!("[{}] {}", label, err.message)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn standalone_source_runs_simple_program() { + execute_source_standalone( + "let x = 41\nlet y = x + 1", + EmbeddedSourceConfig::new("inline.fg", false), + ) + .expect("source should run"); + } + + #[test] + fn standalone_source_rejects_shell_without_permission() { + let err = execute_source_standalone( + r#"sh("echo nope")"#, + EmbeddedSourceConfig::new("shell.fg", false), + ) + .expect_err("shell should be denied"); + + assert!( + err.contains("Shell execution denied"), + "unexpected error: {err}" + ); + } + + #[test] + fn standalone_source_rejects_server_without_routes() { + let err = execute_source_standalone( + "@server(port: 3000)", + EmbeddedSourceConfig::new("server.fg", false), + ) + .expect_err("server without routes should fail"); + + assert!( + err.contains("@server defined but no route handlers found"), + "unexpected error: {err}" + ); + } +} diff --git a/src/runtime/mod.rs b/src/runtime/mod.rs index 9e95c3a..c9c4d87 100644 --- a/src/runtime/mod.rs +++ b/src/runtime/mod.rs @@ -1,4 +1,5 @@ pub mod client; +pub mod embedded; pub mod host; pub mod metadata; pub mod server;