diff --git a/MATH.md b/MATH.md new file mode 100644 index 0000000..81ce9ef --- /dev/null +++ b/MATH.md @@ -0,0 +1,299 @@ +# Mathematical specification of Fpy + +## Scope + +This document defines the semantics of all mathematical actions and builtin functions in Fpy, including: +* Arithmetic +* Truncation, extension +* Casting +* Overflow, underflow +* NaN propagation +* Mathematical library functions + + +## Numerical types + +`U8`, `U16`, `U32`, and `U64` are the primitive unsigned integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard binary representation of unsigned integers. + +`I8`, `I16`, `I32`, and `I64` are the primitive signed integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard two's complement representation of signed integers. + +`F32`, and `F64` are the primitive IEEE floating-point types with bitwidths 32 and 64, respectively. + +## Casts + +A cast is an expression which converts a value `V` of numerical type `S` to a value `R` of numerical type `T`. + +Casts are guaranteed not to end the program, and have no undefined behavior. + + + +If T is a float type, then R is the nearest representable + +WIP +Integer to float and float to float rounds to nearest float (overflow just rounds to highest representable float?) +Float to integer overflow saturates the integer +Integer to integer overflow wraps +The reason arithmetic overflow ends the program but cast overflow does not is because casts are explicit, so the programmer is more aware that they are doing an operation which may involve truncation. If they want to handle an overflow differently than casting does, they can just check for overflow before the cast. + +An intermediate type is picked which is 64 bits. Both operands are coerced to the intermediate type. The result type of the operation, before coercion or casting, is also the intermediate type. + +Arithmetic on F64 +These operations on F64s are fully defined in all cases by IEEE 754 and produce the expected result. These operations never end the program at runtime. + +Arithmetic on I64, U64 +If the operation is division and the denominator is zero, the program ends with an error. + +See https://github.com/rust-lang/rfcs/blob/master/text/0560-integer-overflow.md + +Otherwise, let R be the mathematical result of the operation given the two operands. + +If R is not representable in the result type, the program ends with an error. This is called an underflow or overflow. + + +### `ln` +#### Signature + +`ln(operand: F64) -> F64` + +#### Semantics + +At evaluation: +1. If `operand` is outside the domain of the natural logarithm function, halt the program and display an error code. +2. The expression evaluates to the natural logarithm of `operand`. + +### `iabs` +#### Signature +`iabs(value: I64) -> I64` + +#### Semantics +At evaluation, the function call evaluates to the absolute value of `value`. + +TODO specify what happens if the abs value is outside of i64 + +### `fabs` +#### Signature +`fabs(value: F64) -> F64` + +#### Semantics +At evaluation, the function call evaluates to the absolute value of `value`. +TODO specify what happens if the abs value is outside of i64 + + + +## Type conversion + +TODO Type conversion is the process of converting an expression from one type to another. It can either be implicit, in which case it is called coercion, or explicit, in which case it is called casting. + +### Intermediate types + +The **intermediate type** of a binary or unary operator expression is the type to which all argument expressions will be coerced to. + +Intermediate types are picked via the following rules: + +1. The intermediate type of Boolean operators is always `bool`. +2. The intermediate type of `==` and `!=` may be any type, so long as the left and right hand sides are the same type. If both are numeric then continue. +3. If either argument is non-numeric, raise an error. +4. If the operator is unary `-` and the argument is an unsigned integer, raise an error: negation is undefined for unsigned types (the result would be negative for every nonzero operand). +5. If the operator is `/` or `**`, the intermediate type is always `F64`. +6. If either argument is a float, the intermediate type is `F64`. +7. If either argument is an unsigned integer, the intermediate type is `U64`. +8. Otherwise, the intermediate type is `I64`. + +If the expressions given to the operator are not of the intermediate type, type coercion rules are applied. + +## Result type + +The result type is the type of the value produced by the operator. +1. For numeric operators, the result type is the intermediate type. +2. For boolean and comparison operators, the result type is `bool`. + +Normal type coercion rules apply to the result, of course. Once the operator has produced a value, it may be coerced into some other type depending on context. + + +## Binary operator expressions + +A **binary operator expression** is an expression with a left and right-hand expression, and a binary operator in between, which acts on both values to produce a new value. + +The list of **binary operators** is: +* The [addition operator](#subtraction-semantics) `+` +* The [subtraction operator](#multiplication-semantics) `-` +* The [multiplication operator](#multiplication-semantics) `*` +* The [division operator](#division-semantics) `/` +* The [floor division operator](#floor-division-semantics) `//` +* The [modulus operator](#modulus-semantics) `%` +* The [exponentiation operator](#exponentiation-semantics) `**` +* The [Boolean operators](#boolean-operator-semantics) `and` and `or` +* The [comparison operators](#comparison-semantics) `>`, `>=`, `<`, and `<=` +* The [equality operator](#equality-semantics) `==` +* The [inequality operator](#inequality-semantics) `!=` +* The [range operator](#range-semantics) `..` + +### Syntax + +Rule: + +`binary_op: expr BINARY_OP expr` + +Name: + +`binary_op: lhs op rhs` + +`lhs` and `rhs` are resolved in the value name group. + +### Semantics + +For each use of a binary operator, an [intermediate type](#intermediate-types) is picked, as described in the operator's semantics. + +If `lhs` or `rhs` cannot be [coerced](#type-coercion) into the intermediate type, an error is raised. + +If `lhs` and `rhs` are constant expressions, the binary operator expression is a constant expression. + +At evaluation, for all operators besides the [Boolean operators](#boolean-operator-semantics): +1. `lhs` is evaluated and coerced into the intermediate type. +2. `rhs` is evaluated and coerced into the intermediate type. +3. The expression evaluates to a value of the intermediate type, as described in the operator's semantics. + +#### Addition semantics +The addition operator is `+`. + +If neither `lhs` nor `rhs` are expressions of a [numeric type](#types), an error is raised. + +The expression evaluates to the result of adding + +#### Subtraction semantics +#### Multiplication semantics + +These operators require numeric operands and produce a result in the chosen intermediate type. Addition, subtraction, and multiplication differ only in which arithmetic operation they perform. Integer overflow wraps according to the destination type when the result is ultimately stored, and floating-point operations follow IEEE-754 behavior. + +#### Division semantics +Both operands are promoted to `F64`, and the result is always an `F64`. This means you must explicitly cast the result to store it in an integer type. + +#### Floor division semantics +With integer operands, `//` performs truncating division using the signed or unsigned divide directive. If either operand is a float, the compiler divides in `F64`, converts the quotient to a signed 64-bit integer (which truncates toward zero), and converts back to `F64`, so floating-point floor division also truncates toward zero. + +### Modulus semantics +Modulus works for numeric operands. Signed operands use the signed modulo directive, unsigned operands use the unsigned directive, and floats use floating-point modulo. For signed integers the remainder has the same sign as the dividend. + +#### Exponentiation semantics +Both operands are coerced to `F64`, the exponentiation happens in floating point, and the result type is `F64`. + +#### Boolean operator semantics +Operands must be `bool`. `not` negates a single operand. `and` evaluates the left operand first and only evaluates the right operand when the left operand is `True`. Conversely, `or` skips the right operand when the left operand is `True`. The result of every boolean operator is `bool`. + +#### Comparison semantics +Inequalities require numeric operands. Each operand is coerced to the intermediate type, the comparison runs in that type, and the result is `bool`. + +#### Equality semantics +If both operands are numeric, equality uses the same intermediate-type rules as arithmetic operators. Otherwise both operands must have the exact same concrete type (struct, array, enum, or `Fw.Time`). The compiler compares their serialized bytes. Strings cannot be compared. + +## Unary operators +### Syntax + +Rule: + +`unary_op: expr OP` + +Name: + +`unary_op: val op` + +### Negation operator semantics +### Identity operator semantics + +## Intermediate types + +The **intermediate type** of an operator expression is the type to which the operator's sub-expressions are [coerced](#type-coercion) to. + +If any sub-expression + +### Numeric intermediate types + +The numeric type hierarchy is as follows: +* + + + + # we split this algo up into two stages: picking the type category (float, uint or int), and picking the type bitwidth + + # pick the type category: + type_category = None + if op == BinaryStackOp.DIVIDE or op == BinaryStackOp.EXPONENT: + # always do true division and exponentiation over floats, python style + # this is because, for the given op, even with integer inputs, we might get + # float outputs + type_category = "float" + elif any(issubclass(t, FloatValue) for t in arg_types): + # otherwise if any args are floats, use float + type_category = "float" + elif any(t in UNSIGNED_INTEGER_TYPES for t in arg_types): + # otherwise if any args are unsigned, use unsigned + type_category = "uint" + else: + # otherwise use signed int + type_category = "int" + + # pick the bitwidth + # we only use the arb precision types for constants, so if theyre all arb precision, they're consts + constants = all(t in ARBITRARY_PRECISION_TYPES for t in arg_types) + + if constants: + # we can constant fold this, so use infinite bitwidth + if type_category == "float": + return FpyFloatValue + assert type_category == "int" or type_category == "uint" + return FpyIntegerValue + + # can't const fold + if type_category == "float": + return F64Value + if type_category == "uint": + return U64Value + assert type_category == "int" + return I64Value + + +## Type conversion + +**Type conversion** is the process by which values of one type are converted into values of another type. + +There are two kinds of type conversion: +* [Casting](#casting) +* [Coercion](#type-coercion) + +Type casting is merely an explicit flag for type coercion to take place + + +### Type coercion +**Type coercion** is type conversion that happens implicitly to an expression when required by that expression's semantic context. + + +Coercion happens when an expression of type *A* is used in a syntactic element which requires an expression of type *B*. For example, functions, operators and variable assignments all require specific input types, so type coercion happens in each of these. +In general, the rule of thumb is that coercion is allowed if the destination type can represent all possible values of the source type, with some exceptions. The following rules determine when type coercion can be performed: + +1. If the source and destination types are identical, no coercion is performed. +2. *LiteralString* values may be coerced into any FPP string type. No other string expression can be coerced. +3. Otherwise both source and destination must be numeric (`NumericalValue`). Numeric coercions obey these constraints: + * Floats never coerce to integers. + * Integers may always coerce to floats. + * Float-to-float coercions require a destination bit width greater than or equal to the source width. + * Integer-to-integer coercions require matching signedness and a destination bit width greater than or equal to the source width. + * Arbitrary-precision types (`Int`/`Float`) may coerce to any finite-width numeric type. +If no rule matches, the compiler raises an error. + +Compile-time constant floats (including literals and constant-folded expressions) can only be narrowed into a smaller floating-point type when the value lies inside the destination’s representable range. When the value fits, the compiler rounds it to the nearest representable floating-point number; otherwise compilation fails with an out-of-range error. + + +#### Order of operations +The order in which operations take precedence, from most strongly binding to least strongly binding, is: +1. [Exponentiation](#exponentiation-semantics) +2. [Negation](#negation-operator-semantics) and [identity](#identity-operator-semantics) +3. [Multiplication](#multiplication-semantics), [division](#division-semantics), [floor division](#floor-division-semantics), and [modulus](#modulus-semantics) +4. [Addition](#addition-semantics) and [subtraction](#subtraction-semantics) +5. [Range](#range-semantics) +6. [Comparison](#comparison-semantics) +7. [Not](#boolean-operator-semantics) +8. [And](#boolean-operator-semantics) +9. [Or](#boolean-operator-semantics) + +If two operators have the same precedence in the above list, then the leftmost operator binds more strongly. + diff --git a/MATH_CASTS_DRAFT.md b/MATH_CASTS_DRAFT.md new file mode 100644 index 0000000..1a78753 --- /dev/null +++ b/MATH_CASTS_DRAFT.md @@ -0,0 +1,136 @@ +# Casts — draft normative section for MATH.md + +> Draft to be merged into MATH.md. Mechanized checks: `verify/cast_properties.py`. + +## Value sets + +For each numeric type `T`, the **value set** `⟦T⟧` is the set of values an +expression of type `T` may evaluate to. + +For `n ∈ {8, 16, 32, 64}`: + +* `⟦Un⟧ = { z ∈ ℤ | 0 ≤ z ≤ 2ⁿ − 1 }` +* `⟦In⟧ = { z ∈ ℤ | −2ⁿ⁻¹ ≤ z ≤ 2ⁿ⁻¹ − 1 }` + +For `n ∈ {32, 64}`, `⟦Fn⟧` is the set of IEEE 754 binary-`n` data: + +* all finite values, including the two zeros `+0` and `−0`, +* the two infinities `+∞` and `−∞`, +* a single value `NaN`. + +Fpy does not distinguish NaN payloads and does not distinguish quiet from +signaling NaNs. Every operation that produces a NaN produces the **canonical +NaN** (sign bit 0, quiet bit set, payload 0); a NaN *input* may have any +encoding, and all encodings denote the single value `NaN`. + +> Note: canonical-NaN output matters because struct equality compares +> serialized bytes, so NaN payloads would otherwise be observable through +> `==` on aggregates even though float `==` is numeric. + +## Denotation + +The **denotation** `⟦x⟧` of a value `x` is the mathematical object it +represents. Let `ℝ* = ℝ ∪ {+∞, −∞, NaN}`. + +* If `x` is a value of an integer type: `⟦x⟧ = x ∈ ℤ`. +* If `x` is a finite float value: `⟦x⟧ ∈ ℝ` is the real number assigned by + IEEE 754. In particular `⟦+0⟧ = ⟦−0⟧ = 0` (the denotation is not injective + at zero; see the sign-of-zero clause below). +* `⟦+∞⟧ = +∞`, `⟦−∞⟧ = −∞`, `⟦NaN⟧ = NaN`, as elements of `ℝ*`. + +## Primitive conversion functions + +All casts are composed from four primitives. Each is **total** on its stated +domain; totality of every cast follows. + +**`round_T : ℝ* → ⟦T⟧`** for float `T` (IEEE 754 `convertFormat` / +`convertFromInt` with rounding attribute `roundTiesToEven`): + +1. `round_T(NaN) = NaN` +2. `round_T(+∞) = +∞`, `round_T(−∞) = −∞` +3. `round_T(0) = +0` +4. For nonzero real `r`: the value of `⟦T⟧` nearest to `r`, ties to the one + with even least significant mantissa digit. If `|r|` exceeds the IEEE + overflow threshold for `T`, the result is `+∞` or `−∞` with the sign of + `r` (**not** the largest finite value). If `r` rounds to zero, the result + is `+0` or `−0` with the sign of `r`. + +**`trunc : ℝ* → ℤ ∪ {+∞, −∞, NaN}`** (round toward zero): + +1. `trunc(NaN) = NaN`, `trunc(±∞) = ±∞` +2. For real `r`: the integer with largest magnitude such that + `|trunc(r)| ≤ |r|` and `sign(trunc(r)) ∈ {0, sign(r)}`. + +**`clamp_T : ℤ ∪ {+∞, −∞, NaN} → ⟦T⟧`** for integer `T` with bounds +`T_min`, `T_max`: + +1. `clamp_T(NaN) = 0` +2. `clamp_T(z) = T_min` if `z = −∞` or `z < T_min` +3. `clamp_T(z) = T_max` if `z = +∞` or `z > T_max` +4. `clamp_T(z) = z` otherwise + +**`wrap_T : ℤ → ⟦T⟧`** for integer `T` of bitwidth `n`: the unique +`r ∈ ⟦T⟧` with `r ≡ z (mod 2ⁿ)`. + +## Definition of cast + +A cast converts a value `x ∈ ⟦S⟧` to a value of type `T`. It is the total +function `cast_{S→T} : ⟦S⟧ → ⟦T⟧` defined by exactly one of the following +equations, selected by the kinds of `S` and `T`: + +| `S` | `T` | `cast_{S→T}(x)` | +|---|---|---| +| integer | integer | `wrap_T(x)` | +| integer | float | `round_T(x)` | +| float | float | `±0_T` if `x = ±0_S` (same sign); else `round_T(⟦x⟧)` | +| float | integer | `clamp_T(trunc(⟦x⟧))` | + +The sign-of-zero clause is required because `⟦·⟧` identifies `+0` and `−0`: +float→float casts preserve the sign of zero; float→integer casts map both +zeros to `0`. + +Integer→float casts are a **single direct rounding** into `T`. In +particular, `int → F32` is *not* defined as `int → F64 → F32`; the two +differ for some `|x| > 2⁵³` where the intermediate `F64` result lands +exactly on an `F32` tie (double rounding). + +> Note (rationale): casts never end the program. Arithmetic overflow ends +> the program, but a cast is an explicit request for conversion, so +> truncation/saturation is presumed intended; a program that wants different +> overflow handling can test the value before casting. +> +> Note (correspondence): these semantics coincide with Rust `as`, WASM +> `wrap`/`extend`/`convert`/`promote`/`demote`/`trunc_sat`, LLVM +> `trunc`/`sext`/`zext`/`sitofp`/`uitofp`/`fpext`/`fptrunc`/ +> `llvm.fptosi.sat`/`llvm.fptoui.sat`, and Java primitive conversions. +> They do **not** coincide with LLVM's plain `fptosi`/`fptoui` (poison on +> out-of-range) or WASM's plain `trunc` (traps). + +## Theorems + +The following hold for all numeric `S`, `T` and all `x ∈ ⟦S⟧` +(mechanically checked in `verify/cast_properties.py`): + +* **T1 (totality, determinism).** `cast_{S→T}` assigns exactly one value of + `⟦T⟧` to every `x ∈ ⟦S⟧`. *(By construction: each equation is a + composition of total functions with exhaustive, mutually exclusive + cases.)* +* **T2 (identity).** `cast_{T→T}(x) = x`. +* **T3 (widening exactness).** If `⟦x⟧` is exactly representable in `T`, + then `⟦cast_{S→T}(x)⟧ = ⟦x⟧`. Consequences: `F32→F64` is exact; + integer→integer with same signedness and greater width is exact; + integer→float is exact when the integer fits in the mantissa + (`|x| ≤ 2²⁴` for `F32`, `|x| ≤ 2⁵³` for `F64`). +* **T4 (saturation boundaries).** For float→integer: + `cast(NaN) = 0`, `cast(+∞) = T_max`, `cast(−∞) = T_min`, `cast(±0) = 0`, + and for finite `x`, `cast(x) ∈ [T_min, T_max]` with equality at the + bounds exactly when `trunc(⟦x⟧)` is out of range on that side. +* **T5 (monotonicity).** Float→integer and integer→float casts are monotone + on non-NaN inputs: `x ≤ y ⇒ cast(x) ≤ cast(y)`. +* **T6 (round trips).** `S→T→S` is the identity when every value of `S` is + exactly representable in `T` (e.g. `I32→F64→I32`). It is **not** the + identity otherwise (e.g. `I64→F64→I64`, `I32→F32→I32`); the checker + exhibits counterexamples. +* **T7 (no double rounding).** Direct `U64→F32` differs from + `U64→F64→F32` for some inputs (the checker exhibits one); for `I32` and + narrower sources the two coincide. The spec mandates the direct form. diff --git a/MATH_COMPARISON.md b/MATH_COMPARISON.md new file mode 100644 index 0000000..f06fb05 --- /dev/null +++ b/MATH_COMPARISON.md @@ -0,0 +1,252 @@ +# Fpy arithmetic semantics vs Rust and C# + +How fpy's arithmetic rules (MATH.md, mechanized in `verify/arith_properties.py`) +compare to Rust and C#. **Every Rust and C# claim in this document was verified +empirically** by compiling and running probe programs; the raw outputs are in +the appendix. + +Verified with: + +* Rust: `rustc 1.96.0`, run twice: default profile (debug assertions ON) and + `-O` (debug assertions OFF), on x86-64 Linux. +* C#: .NET SDK `8.0.422` (RyuJIT, x86-64 Linux), default `unchecked` context + plus explicit `checked` expressions. + +## Integer arithmetic + +| Behavior | fpy | Rust | C# | +|---|---|---|---| +| `+ - *` overflow | halts, always | debug: panics; release: wraps (still defined as a bug; `checked_*`/`wrapping_*` express intent) | unchecked (default): wraps; `checked`: throws `OverflowException` | +| divide by zero | halts (`DOMAIN_ERROR`) | panics, **both profiles** | throws `DivideByZeroException`, always | +| remainder by zero | halts (`DOMAIN_ERROR`) | panics, both profiles | throws `DivideByZeroException`, always | +| `MIN / -1` (fpy `//`) | halts (`ARITHMETIC_OVERFLOW`) | panics, both profiles | throws `OverflowException`, **even unchecked** | +| `MIN % -1` | halts (`ARITHMETIC_OVERFLOW`) | panics, both profiles (`checked_rem` returns `None`) | throws `OverflowException`, even unchecked | +| division style | `//` floors toward -inf, `%` takes divisor's sign (Python) | `/` truncates toward zero, `%` takes dividend's sign (C); `div_euclid`/`rem_euclid` as methods | `/` truncates, `%` takes dividend's sign (C) | +| `/` on integers | always computes in F64 (true division) | stays integer | stays integer | + +Notes: + +* Both Rust and C# treat `+ - *` overflow checking as a *mode* (debug profile, + `checked` context) but check division unconditionally: div-by-zero and + `MIN/-1` are hard errors in every configuration, because the underlying + hardware/LLVM operation is undefined there. fpy's always-halt rule for + `+ - *` is Rust's debug behavior made permanent, which fpy's VM already + implements (`ARITHMETIC_OVERFLOW`/`UNDERFLOW`); the LLVM backend still wraps + (open question OQ-1 in `verify/arith_properties.py`). +* `MIN % -1` halting (rather than evaluating to 0) was decided 2026-07-06 to + match Rust and C#: **both** error here in every mode, even though the + mathematical remainder is 0. It also keeps `//` and `%` halting on exactly + the same inputs, so `a == (a // b) * b + (a % b)` holds wherever the pair is + defined (proved in `verify/arith_properties.py`). +* fpy is alone in floored division. Rust and C# are C-style truncating. fpy + follows Python because sequences are written by Python users; the VM, + the LLVM backend, and the spec all agree on floored. + +## Type discipline + +| Behavior | fpy | Rust | C# | +|---|---|---|---| +| unary `-` on unsigned | **compile error** (as of 2026-07-06) | compile error `E0600` (`wrapping_neg()` expresses intent) | `uint`: legal, **promotes to `long`** (result is correct, e.g. `-5`); `ulong`: compile error `CS0023` | +| mixed-width same-signedness (`u8 + u64`) | implicit widening to the 64-bit intermediate | compile error `E0277` (no implicit conversions at all) | implicit promotion (both sides widen) | +| mixed signedness (`i32 + u32`) | compile error | compile error `E0277` | promotes both to `long`; `ulong + int` is compile error `CS0034` (no type holds both) | +| int -> float implicit | allowed (RNE rounding, can be lossy for wide ints) | compile error `E0308` (`as` required) | allowed, even lossy `long -> float` | +| narrowing implicit (`i64 -> i32`) | compile error | compile error | compile error `CS0266` | + +Notes: + +* fpy sits between the two: stricter than C# (no signed/unsigned mixing) but + looser than Rust (implicit widening and int->float are allowed because the + 64-bit intermediate makes them value-preserving or explicitly rounding). +* C#'s `-uint -> long` promotion is the mathematically honest alternative to + rejecting unary minus on unsigned. fpy chose Rust's rule (reject) instead: + fpy's unsigned intermediate would have been U64, where C#'s trick has no + wider signed type to escape to -- exactly why C# rejects `-ulong`. + +## Floating point + +| Behavior | fpy | Rust | C# | +|---|---|---|---| +| `NaN != NaN` | True | true | True | +| `NaN == NaN`, `NaN < x` | False | false | False | +| `1.0 / 0.0` | +inf, no halt | inf, no panic | Infinity, no throw | +| `0.0 / 0.0` | NaN | NaN | NaN | +| float `%` style | floored, sign of divisor (`-7.5 % 2.0 == 0.5`, Python) | truncated fmod, sign of dividend (`-7.5 % 2.0 == -1.5`, C) | truncated fmod, sign of dividend (`-1.5`, C) | +| float `% 0.0` | NaN (never halts) | NaN, no panic | NaN, no throw | +| overflow to +-inf, subnormals | IEEE-754 | IEEE-754 | IEEE-754 | + +Note on `% 0.0` (OQ-5, resolved 2026-07-06): Rust and C# both give NaN -- +float ops never trap in either language -- and fpy follows them: the spec, +the LLVM backend (frem), and the VM model all produce NaN. CPython raises +`ZeroDivisionError` instead; this is a deliberate divergence from Python. +The C++ `op_fmod` still returns DOMAIN_ERROR and needs the upstream fix +below. + +## Numeric casts + +| Behavior | fpy cast | Rust `as` | C# cast | +|---|---|---|---| +| int -> smaller int | wrap (truncate bits) | wrap (`-1i8 as u8 == 255`, `300i64 as u8 == 44`) | unchecked: wrap (`(byte)300 == 44`); checked: throws | +| float -> int, in range | truncate toward zero | truncate | truncate | +| float -> int, out of range | saturate to MIN/MAX | saturate (`300.0 as u8 == 255`, `-300.0 as i8 == -128`) | unchecked: **unspecified value** (measured on x64 .NET 8: sentinel `int.MinValue` for `(int)3e10`); checked: throws | +| NaN -> int | 0 | 0 | unchecked: measured `int.MinValue` (unspecified per spec); checked: throws | +| int -> float | round to nearest (RNE) | round to nearest | round to nearest | + +# TODO i think we might want to follow C# for the Nan-> int and float->int out of range + +fpy's cast spec (MATH_CASTS_DRAFT.md) is exactly Rust's `as` semantics -- Rust +moved float->int to saturating in 1.45 to remove the same UB fpy avoids. C# is +the outlier: its unchecked out-of-range float->int is explicitly "an +unspecified value of the destination type" (memory-safe UB), and on x64 .NET 8 +it produces a sentinel, not saturation. + +## User exit vs runtime fault + +Question: should a user-invoked `exit(code)` be distinguishable from a runtime +arithmetic fault, so a sequence cannot fake (or accidentally collide with) a +`DOMAIN_ERROR`? Precedent says yes, with one caveat about *where* the +distinction lives: + +* **Rust** separates *in process*: a panic runs the panic hook, prints + diagnostics, and can be observed by `catch_unwind`; `std::process::exit(n)` + does none of that. But at the OS level the channels collapse to one integer: + a panicking process exits with code 101, and `process::exit(101)` is + indistinguishable to the parent (verified). The separation is real only + because supervisors look at the panic diagnostics/hook, not the code. +* **C#** likewise: an unhandled exception has its own termination path, + diagnostics, and type; `Environment.Exit(n)` is just a code. In process the + channels are distinct; the integer alone is spoofable. +* **F Prime C++ FpySequencer already implements the separation correctly**: + `exit_directiveHandler` raises a dedicated event + (`SequenceExitedWithError(path, userCode)`) carrying the user's code, and + reports the directive error as `EXIT_WITH_ERROR` -- always. A user exit can + never surface as `DirectiveError::DOMAIN_ERROR`; the fault enum values are + reserved for actual faults. +* The **Python VM model** also keeps them apart: `handle_exit` sets + `error_code` (user-owned I32) and returns no directive error; faults return + a `DirectiveErrorCode` from the handler. +* The **LLVM/wasm backend** used to be the one place they were conflated (a + single `fpy_exit(i32)` host import shared by `exit()`, `assert`, and the + arithmetic guards). As of 2026-07-06 the wasm ABI has two noreturn host + imports: `fpy_exit(user_code)` for user-requested termination (exit(), + assert) and `fpy_fault(directive_error)` for runtime faults (division by + zero, arithmetic overflow). The test runner reports them as distinct + outcomes (`exit ` vs `fault `), and the test helpers refuse + cross-channel matches: `exit(10)` does not satisfy an expected + `DOMAIN_ERROR` even though `DOMAIN_ERROR`'s value is 10. + +The general lesson (which the Rust exit-code collision demonstrates): the +separation must live in the *channel*, not in an integer convention. + +## Known divergences to fix upstream (C++ FpySequencer) + +Found while verifying, in `Svc/FpySequencer/FpySequencerDirectives.cpp`: + +1. `op_sdiv` computes `lhs / rhs` with only a zero-divisor guard: + `INT64_MIN / -1` is C++ UB (SIGFPE on x86). Needs the overflow guard the + fpy model/backends now have. +2. `op_smod` computes `lhs % rhs` with only a zero-divisor guard: + `INT64_MIN % -1` is likewise UB. Same guard needed + (Rust/C# both error here; the fpy spec now halts with + `ARITHMETIC_OVERFLOW`). +3. `op_fmod` returns `DOMAIN_ERROR` on a zero divisor, but the fpy semantics + (spec, LLVM backend, VM model, Rust, C#, IEEE) is NaN with no halt. It + also computes `lhs - rhs * floor(lhs / rhs)`, which rounds at every step; + the spec's formula is exact truncated fmod plus at most one rounded + addition of the divisor (what `frem` + `fadd` and the VM model compute), + and the two differ in the last ulp for extreme operand ratios. +4. `exit_directiveHandler` pops the exit code as `U8`, while the fpy compiler + and Python model treat it as `I32` -- worth checking version alignment. + +## Appendix: probe outputs + +### Rust runtime (`rustc 1.96.0`) + +Left: default build (debug assertions on). Right: `-O` (off). Lines identical +between profiles are shown once. + +``` + debug release +i64_add_overflow_panics: true false +u64_sub_underflow_panics: true false +i64_mul_overflow_panics: true false +i64_div_by_zero_panics: true true +i64_rem_by_zero_panics: true true +i64_min_div_neg1_panics: true true +i64_min_rem_neg1_panics: true true +checked_rem_min_neg1: None +trunc_div_7_by_neg2: -3 +trunc_div_neg7_by_2: -3 +rem_neg7_by_2: -1 +rem_7_by_neg2: 1 +div_euclid_neg7_by_2: -4 +rem_euclid_neg7_by_2: 1 +wrapping_neg_5u32: 4294967291 +nan_ne_nan: true +nan_eq_nan: false +nan_lt_1: false +f64_1_div_0: inf +f64_0_div_0: NaN +f64_rem_neg7p5_by_2: -1.5 +f64_rem_7p5_by_neg2: 1.5 +f64_rem_1_by_0: NaN +as_sat_300f64_to_u8: 255 +as_sat_neg300f64_to_i8: -128 +as_nan_to_i32: 0 +as_wrap_neg1i8_to_u8: 255 +as_trunc_300i64_to_u8: 44 +as_2pow63_f64_to_i64_saturates: true +panic exit code: 101 +process::exit(101) exit code: 101 (indistinguishable to the parent) +``` + +Compile errors (all rejected): + +``` +-x where x: u32 error[E0600]: cannot apply unary operator `-` to type `u32` +i32 + u32 error[E0277]: cannot add `u32` to `i32` +u8 + u64 error[E0277]: cannot add `u64` to `u8` +let _: f64 = 1i64 error[E0308]: mismatched types +``` + +### C# runtime (.NET SDK 8.0.422, x64, default unchecked) + +``` +unchecked_add_wraps_to_min: True +checked_add: OverflowException +div_by_zero: DivideByZeroException +rem_by_zero: DivideByZeroException +min_div_neg1_unchecked: OverflowException +min_rem_neg1_unchecked: OverflowException +trunc_div_7_by_neg2: -3 +trunc_div_neg7_by_2: -3 +rem_neg7_by_2: -1 +rem_7_by_neg2: 1 +neg_uint_type: Int64 value -5 +uint_plus_int_type: Int64 value -1 +implicit_long_to_float: 9.223372E+18 +implicit_long_to_double: 9.223372036854776E+18 +nan_ne_nan: True +nan_eq_nan: False +nan_lt_1: False +f64_1_div_0: Infinity +f64_0_div_0: NaN +f64_rem_neg7p5_by_2: -1.5 +f64_rem_7p5_by_neg2: 1.5 +f64_rem_1_by_0: NaN +unchecked_double300_to_byte: 44 +unchecked_doubleNeg300_to_sbyte: -44 +unchecked_double3e10_to_int: -2147483648 +unchecked_nan_to_int: -2147483648 +checked_double3e10_to_int: OverflowException +unchecked_int300_to_byte: 44 +checked_int300_to_byte: OverflowException +long_to_double_rounds: True +``` + +Compile errors (all rejected): + +``` +-x where x: ulong error CS0023: Operator '-' cannot be applied to operand of type 'ulong' +ulong + int error CS0034: Operator '+' is ambiguous on operands of type 'ulong' and 'int' +int x = (long)y implicit error CS0266: Cannot implicitly convert type 'long' to 'int' +``` diff --git a/MATH_TODO.txt b/MATH_TODO.txt new file mode 100644 index 0000000..1c0fb46 --- /dev/null +++ b/MATH_TODO.txt @@ -0,0 +1,111 @@ +MATH_TODO: formal spec for numeric cast semantics +================================================== + +Goal +---- +Write a consistent, machine-checked spec for casting between numeric types +(U8-U64, I8-I64, F32, F64). "No undefined behavior" == the cast semantics is a +total, deterministic function: for every (source type S, target type T, value +v : S) the spec assigns exactly one result. Spec is written from scratch and is +normative; the current compiler, SPEC.md, MATH.md, and backends are NOT +authoritative and will be brought into line afterward. + +Chosen semantics (decided) +-------------------------- +Adopt the industry-consensus saturating conversion semantics. This is +simultaneously: WASM trunc_sat / promote / demote / convert, LLVM +llvm.fptosi.sat / fptoui.sat + fpext/fptrunc/sitofp/uitofp, Rust `as`, and +Java. LLVM and WASM do not meaningfully diverge on conversions once the +saturating variants are chosen; the only ecosystem split is trap-on-overflow +(WASM default trunc) vs saturate (everything else). We saturate, consistent +with the principle that casts never end the program. Never adopt LLVM's +default fptosi/fptoui -- out-of-range is poison, i.e. exactly the UB we are +eliminating. + +Per conversion class: +* int -> int: wrap (reduce mod 2^n, reinterpret as two's complement if T + signed). Total, no edge cases. +* int -> float: IEEE round-to-nearest-even, as a SINGLE direct rounding to + the target float type (int -> F32 does not go via F64; avoids double + rounding for |v| > 2^53 landing on F32 ties). Total. +* float -> float: widen is exact; narrow is IEEE RNE, overflow -> +/-inf + (NOT max-finite; resolves the open "?" in MATH.md draft). Total. +* float -> int: truncate toward zero, then clamp to [T::MIN, T::MAX]. + NaN -> 0. -0.0 -> 0. +/-inf clamp to max/min. Total, never traps. + +NaN policy: ignore signaling NaNs (like WASM). Stronger than WASM for +determinism: any operation producing NaN produces THE canonical quiet NaN +(sign 0, quiet bit set, payload 0). This matters because fpy struct equality +compares serialized bytes, so NaN payloads are observable through == on +structs even though float == is numeric. NaN inputs may be any NaN. + +Decided: int -> F32 is a direct single rounding +----------------------------------------------- +Spec = direct single RNE to F32 (what sitofp/convert do; standard). NOT the +composition int -> F64 -> F32: two roundings can double-round differently for +|v| > 2^53 when the F64 result lands exactly on an F32 tie. LLVM/WASM backends +already do direct conversion; the VM needs SITOFP/UITOFP variants targeting +F32 (+2 opcodes, see below). The Z3 harness should still encode both functions +and confirm they differ (produces the tie counterexample) as a sanity check. + +Current state (why change is needed) +------------------------------------ +* FpySequencer VM (../fprime-fpy-testbed/fprime/Svc/FpySequencer/ + FpySequencerDirectives.cpp, floatToWrappedIntBits): float->int truncates + then wraps mod 2^64, but returns DOMAIN_ERROR (sequence death) for NaN, + +/-inf, or magnitude >= 2^64. Matches no ecosystem semantics (I64(1e20) + wraps to garbage, I64(1e300) kills the sequence). Replace. +* Backends diverge today on float -> narrow int: LLVM saturates at target + width (fptosi.sat.i8: I8(300.0) == 127); bytecode saturates/wraps at 64-bit + then wrap-truncates (I8(300.0) == 44). Spec (saturate at target width) + ratifies the LLVM behavior; bytecode path must change. + +Bytecode (VM) changes +--------------------- +* Replace FPTOSI/FPTOUI with FTOI_SAT_{8,16,32,64}_{S,U} (8 opcodes, net +6): + pop F64, trunc toward zero, clamp to target range, NaN -> 0, push + sign/zero-extended to 64 bits. ~6 lines of C++ each; the mod-2^64 wrapping + logic goes away. DOMAIN_ERROR disappears from casts entirely. +* Add SITOFP_32/UITOFP_32 (or equivalent) for direct int -> F32 (+2 opcodes), + per the direct-rounding decision above. Result pushed as F32 extended to the + 64-bit stack slot via FPEXT semantics is NOT acceptable if it re-rounds; + push the F32 bit pattern / value such that the F32 result is exact. (An + fpext of the already-rounded F32 back to F64 is exact, so extending the + *result* to keep the 64-bit stack model is fine -- the rounding to F32 must + simply happen first.) +* Keep unchanged: FPEXT, FPTRUNC, SITOFP, UITOFP, SIEXT_*, ZIEXT_*, ITRUNC_* + (ITRUNC still correct for wrapping int->int casts). +* Rejected alternative: zero VM changes by emitting inline compare-and-clamp + before ITRUNC -- bloats every narrow cast site in uplinked sequences; + dedicated opcodes are easier to audit. +* LLVM backend: already correct (sat intrinsics). WASM backend: use + iN.trunc_sat_f64_s/u. Both get the spec essentially for free. + +Spec-writing plan +----------------- +1. Rewrite the Casts section of MATH.md denotationally, WASM-spec style: + define value sets (integers as subsets of Z; floats as finite IEEE values + plus +/-inf and canonical NaN), then two primitives round_T (RNE into a + float type) and clamp_T / wrap_T (into an int type), then each cast as a + total function composed from them. Crib definitional style from the WASM + Core spec numerics section (it is mechanized in Isabelle/Coq and already + written as total functions). +2. Mechanize in Z3 (Python bindings; native bitvector + IEEE-754 FloatingPoint + theories; everything finite so decidable/push-button). Encode each cast as + an ite-tree over total SMT ops -- total and deterministic by construction. + GOTCHA: SMT-LIB's fp.to_sbv/fp.to_ubv are themselves underspecified for + NaN/out-of-range; define our own total wrappers, use built-ins only on + their defined domain. +3. Check theorems for all (S, T) pairs: + - range soundness: result in [T::MIN, T::MAX] (the clamp actually clamps) + - identity: cast T -> T == id + - value preservation: if v exactly representable in T, cast preserves the + denoted real number (catches truncate-vs-round bugs) + - monotonicity of float->int and int->float on non-NaN inputs + - widening round-trips: I32 -> F64 -> I32 == id (holds); the checker + SHOULD produce a counterexample for I64 -> F64 -> I64 (that's it working) +4. Belt and suspenders: exhaustive enumeration for 8/16-bit types and + F32 -> * (2^32 inputs) against a reference implementation, in pytest. +5. Later / out of scope for now: per-backend "implements spec" checking + (translation validation); fixing the VM in fprime-fpy-testbed; arithmetic, + builtins (ln, iabs, fabs), overflow-traps-on-arithmetic rationale. diff --git a/README.md b/README.md index 732f366..f32cde7 100644 --- a/README.md +++ b/README.md @@ -635,7 +635,7 @@ This project uses [uv](https://docs.astral.sh/uv/) to manage its environment. The hooks are defined in `.pre-commit-config.yaml` and run automatically on `git commit` once installed: * **black** formats the staged Python files. -* **spec test links** runs `verify/spec_links.py`, which checks that every test link in `SPEC.md` (the `*Tests:*` lines) points at a test that exists, at its current line number. If this hook fails because tests moved or were renamed, run `uv run python verify/spec_links.py --fix` to update the line numbers and link labels, then re-stage `SPEC.md`. A missing or renamed test must be fixed by hand. +* **spec test links** runs `verify/spec_links.py`, which checks that every test link in the spec sources under `docs/spec/` (the `_Tests:_` lines) points at a test that exists, at its current line number. If this hook fails because tests moved or were renamed, run `uv run python verify/spec_links.py --fix` to update the line numbers and link labels, then re-stage the changed `docs/spec/*.adoc` files. A missing or renamed test must be fixed by hand. You can run all hooks against the whole repo at any time with `uv run pre-commit run --all-files`. diff --git a/SPEC.md b/SPEC.md deleted file mode 100644 index 12c1e4c..0000000 --- a/SPEC.md +++ /dev/null @@ -1,1373 +0,0 @@ -> **WARNING:** The Fpy specification is a work-in-progress - -# Fpy Specification - -Fpy is a sequencing language for the F-Prime flight software framework. It combines Python-like syntax with the FPP model, with domain-specific features for spacecraft operation. - -This document specifies the syntax and semantics of the Fpy sequencing language. In other words, it specifies which programs the compiler should accept, and how the output should behave when run. The intent is to leave the implementation of the compiler, the bytecode it generates, and the virtual machine it runs on, unspecified. - -It is assumed the reader is familiar with the FPP model, including commands, telemetry, parameters, structs, arrays and enums. - -> Informal notes and explanations are quoted like this. - -Terms are **bolded** in the location they are primarily defined. - -`Monospaced` text refer to syntactic rules, type names or example Fpy code. - -In a syntactic rule: -* Text in between forward slashes `/` is regex -* Text in between square brackets `[]` is optional -* Text in between parentheses `()` is handled as a group -* A plus suffix `+` means one or more instances of its preceding rule -* A star suffix `*` means zero or more instances of its preceding rule -* A question mark suffix `?` means zero or one instances of its preceding rule - -A line of the specification may be followed by an italicized *Tests:* line. Each bracketed number links to a test that verifies the behavior; hover over a number to see the test's full `file::class::name`. These links are checked and kept up to date by `verify/spec_links.py`. - -# Names and scopes - -## Names - -`name: /\$?[^\W\d]\w*/` - -A **name** is a string consisting of letters, underscores or digits. The first character may not be a digit. - -The first character may optionally be `$`, in which case the name is considered **escaped**. An escaped name is the same as an unescaped name, except in that it always lexes as a name, even if it is a [reserved word](#reserved-words). - -## Reserved words - -A **reserved word** is a word which cannot be used as an [unescaped name](#names). - -The list of reserved words is: -* `assert` -* `break` -* `check` -* `continue` -* `def` -* `for` -* `if` -* `import` -* `not` -* `pass` -* `return` -* `while` - -## Symbols - -A **symbol** is a language construct that can be referred to by a name in the program. - -The following language constructs may be symbols: -* [namespaces](#namespaces) -* [variables](#variables) -* [callables](#callables) -* [types](#types) -* telemetry channels -* parameters -* enum constants -TODO members? -TODO you're selecting a member of a value--not referring to a static member. when you say a.b you're asking for a comp to be performed - -## Scopes - -A **scope** is a mapping of names to symbols, accessed via some region of the source code. -TODO a scope IS a REGION - -The **global scope** is the scope accessible throughout the entire source code. - -Each [function](#functions) has a **function scope**, accessible in its body. - -The **resolving scope** is the most specific scope that some part of the source code has access to. - -Scopes may have a **parent scope**: -* The parent scope of a function scope is the global scope. -* The global scope does not have a parent scope. - -## Name groups - -Scopes are divided into **name groups**. - -The list of name groups is: -* The **[value](#values) name group** -* The **[type](#types) name group** -* The **[callable](#callables) name group** - -Each name group only contains names which map to their particular language construct, or namespaces with the same property, recursively. -TODO Maybe could remove this line - -TODO explain why we need this -Name groups do not intersect. - -> This means that the names of callables, types and values never conflict. WITH EACH OTHER - -Name groups are accessed via syntactic context. -TODO this is really just an explanation - -TODO type names should NOT be expressions - - -TODO A.b is an expr, A is an expr, b is not an expression. A.b is a dot expression. b is not an expression, it's part of a -TODO Does it refer to something that has a scope, or does it refer to something that has members. - -> For instance, the type name group is accessible anywhere in the source code where a type name is expected, such as a [variable definition](#variable-definition) type annotation, or a [function definition](#function-definition) return type. - -The **resolving name group** is the name group that a name should be resolved in, based on its syntactic context. - -## Namespaces - -A **namespace** is a mapping of names to symbols, associated with a name. - -## Qualified names - -A **qualified name** is one of: -* A name -* A qualified name, followed by a `.`, followed by a name - -The **qualifier** is the qualified name to the left of the `.`. - -To resolve a qualified name in a name group: -1. If there is no qualifier: - 1. Resolve the name in the resolving scope. - 2. If the name fails to be resolved, resolve the name in the parent scope of the resolving scope. -2. Otherwise: - 1. Resolve the qualifier. - 2. If the qualifier is an expression, resolution is handled by the rules of [member access](#member-access-expression). - 3. If the qualifier is not a namespace, an error is raised. - 4. Resolve the name in the qualifier namespace. - -If at any point a name fails to be resolved, an error is raised, unless otherwise specified. - -A **fully-qualified name** is a qualified name which is not itself a qualifier. -TODO names are semantic, ident is syntactic -TODO you can't actually tell at syntax level what is a fqn -If a fully-qualified name resolves to a namespace, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L343 "test/fpy/test_imports.py::TestImportNamespaceIsolation::test_module_name_not_usable_as_value") - -TODO you can think of the dict as "importing definitions" - - -> Namespace symbols cannot be used anywhere, so this forces names to resolve to something "useful" - -TODO I'm not sure this is clear what this means. The idea here is that the full qualified name should always reference SOMETHING--cannot just put Svc in place of a type, even though Svc does resolve in type name group and global scope. - -## Definitions - -A **definition** is a language construct that introduces a name-to-[symbol](#symbols) mapping as apart of a [scope](#scopes) and [name group](#name-groups). - -The list of definitions is: -* [Variable definitions](#variable-definition) -* [Function definitions](#function-definition) - -# Variables - -A **variable** is a symbol with a static [type](#types) and a dynamic, mutable [value](#values). -todo maybe call it dynamic? - -TODO: give a list of statements - -## Variable definition - -A **variable definition statement** introduces a name-to-variable mapping in its resolving scope. - -### Syntax - -Rule: - -`variable_declare_stmt: name ":" qualified_name "=" expr` - -Name: - -`variable_declare_stmt: lhs ":" type_ann "=" rhs` - -`lhs` and `rhs` are resolved in the value name group. - -`type_ann` is resolved in the type name group. - -### Semantics - -If `lhs` resolves to a previously-defined symbol, an error is raised. - -> This prevents redefining a variable. - -If `rhs` cannot be coerced to type `type_ann`, an error is raised. - -The new variable has type `type_ann`. It is added to the resolving scope under name `lhs`. - -At execution, `rhs` is [evaluated](#evaluation) and [coerced](#type-coercion) to type `type_ann`. This becomes the variable's initial value. - -For statements following this, the variable `lhs` is considered defined. - -## Variable assignment - -A **variable assignment statement** mutates the value of a variable. - -### Syntax -Rule: - -`variable_assign_stmt: name "=" expr` - -Name: - -`variable_assign_stmt: lhs "=" rhs` - -`lhs` and `rhs` are resolved in the value name group. - -### Semantics - -If `lhs` does not resolve to a variable, an error is raised. - -> Hereafter, `lhs` refers to the variable named `lhs`. - -If `lhs` has not been defined yet, an error is raised. - -If `rhs` cannot be coerced to `lhs`'s type, an error is raised. - -At execution, `rhs` is [evaluated](#evaluation) and [coerced](#type-coercion) to `lhs`'s type. This becomes the `lhs`'s new value. - -## Member assignment - -A **member assignment statement** mutates the value of a [member](#structs) in a variable. - -### Syntax -Rule: - -`variable_assign_member_stmt: expr "." name "=" expr` - -Name: - -`variable_assign_member_stmt: parent "." member "=" rhs` - -`parent`, and `rhs` are resolved in the value name group. - -### Semantics - -If `parent` is not a variable or a [field](#fields) with a [field base](#fields) that is a variable, an error is raised. - -> This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you're modifying is a variable. - -If `parent`'s type is: -* not [constant-sized](#types), or -* not a [struct](#structs) type, or -* `member` is not a member of `parent`'s type - -... an error is raised. - -If the variable has not been defined yet, an error is raised. - -If `rhs` cannot be coerced to the member's type, an error is raised. - -At execution, `rhs` is [evaluated](#evaluation) and [coerced](#type-coercion) to the member's type. This becomes the member's new value. - -The value of the variable is unchanged except for the member. - -## Element assignment - -An **element assignment statement** mutates the value of an [element](todo) in a variable. - -### Syntax -Rule: - -`variable_assign_element_stmt: expr "[" expr "]" "=" expr` - -Name: - -`variable_assign_element_stmt: parent "[" item "]" "=" rhs` - -`parent`, `item` and `rhs` are resolved in the value name group. - -### Semantics - -If `parent` is not a variable or a [field](#fields) with a [field base](todo) that is a variable, an error is raised. - -> This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you're modifying is a variable. - -If the variable has not been defined yet, an error is raised. - -If `item` cannot be [coerced](#type-coercion) to [array index type](#type-aliases), an error is raised. - -If `parent`'s type is: -* not [constant-sized](#types), or -* not an [array](#arrays) type, or -* `item` is a [constant](todo) with a value less than 0 or greater than the `parent`'s type length, - -... an error is raised. - -If `rhs` cannot be coerced to the element's type, an error is raised. - -At execution: -1. `rhs` is evaluated and coerced to the element's type -2. `item` is [evaluated](#evaluation) and coerced to [array index type](#type-aliases) -3. If `item` is less than zero or greater than the `parent`'s type length, a runtime error is raised -4. The element in the `parent` array at the index `item` is set to the result of step 1 - -The value of the variable is unchanged except for the element. - -## Variable evaluation - -The value produced by [evaluating](#expressions) a variable is the value most recently assigned to that variable, or the initial value if it has only been defined. - -If a variable is evaluated before it has been defined, an error is raised. - -# Functions - -A **function** is a [callable](#callables) [symbol](#symbols) with an inner scope, parameters, code and a return [type](#types). - -The **call site** is the location in the source code at which a function is called. - -## Function parameters - -A **function parameter** is a [variable](#variables) implicitly defined by a function in that function's scope. - -When a function is [called](todo), each parameter is set to an initial value. - -The initial value may either be from a passed [argument](todo), or a default value, if one is specified in the function definition. - -> In all other respects, parameters are like normal variables, meaning you can modify them in the function body. - -## Return types - -The **return type** of a function is the [type](#types) of the value returned by that function. If the return type is [Nothing](#internal-types), the function does not return a value. - -## Returns - -A **return statement** ends the currently executing function, resumes execution at the call site, and optionally returns a value. - -### Syntax -Rule: - -`return_stmt: "return" [expr]` - -Name: - -`return_stmt: "return" value` - -`value` is resolved in the value name group. - -### Semantics - -If the return statement is outside of a function body, an error is raised. - -The **enclosing function** of a return statement is the function whose body that return is in. - -If `value` is not provided and the enclosing function's return type is not Nothing, an error is raised. - -If `value` is provided and cannot be [coerced](#type-coercion) to the return type of the enclosing function, an error is raised. - -At execution: -1. If provided, `value` is [evaluated](todo) and [coerced](#type-coercion) to the return type of the enclosing function. -2. The execution of the function body is stopped, and execution after the function call site resumes. - -## Function definition - -A **function definition statement** introduces a name-to-[function](#function) mapping in the global scope. - -### Syntax - -Rule: - -``` -function_def_stmt: "def" name "(" [parameters] ")" ["->" qualified_name] ":" block -parameters: parameter ("," parameter)* -parameter: name ":" qualified_name ["=" expr] -``` - -Name: - -``` -function_def_stmt: "def" name "(" parameters ")" "->" return_type ":" body -parameters: parameter_0 "," parameter_1 ... "," parameter_n -parameter: parameter_name ":" parameter_type "=" parameter_default_value -``` - -A function definition statement is only valid outside an indentation block. - -The parameter `name`s are resolved in the value name group. - -`name` is resolved in the callable name group. - -`return_type` and each of the parameter types are resolved in the type name group. - -### Semantics - -If `name` resolves to a previously-defined callable, an error is raised. - -A new function [scope](#scopes) is created, accessible to the `body` and the parameter `name`s. - -Each parameter is a variable in this new scope. - -> This implies that no two parameters may have the same name, otherwise they would be conflicting variables. - -If the default value of a parameter is not a [constant](todo), an error is raised. - -If the default value of a parameter cannot be [coerced](#type-coercion) to the type of the parameter, an error is raised. - -If a parameter without a default value follows a parameter with a default value, an error is raised. - -If `return_type` is provided, and any [branch](todo) of the function does not return a value, an error is raised. -TODO need a section on control flow? - -The new function with name `name` is added to the global scope. If `return_type` is not provided, the [return type](#return-types) is [Nothing](#internal-types), otherwise the return type is type `return_type`. - -> Because functions can only be defined in the global scope, you cannot declare a function in a function. - -> Functions can be used before they are defined. - -# Ifs -An **if statement** conditionally executes blocks of code. - -## Syntax -Rule: - -`if_stmt: "if" expr ":" stmt_list elifs ["else" ":" stmt_list]` - -`elifs: elif_*` - -`elif_: "elif" expr ":" stmt_list` - -Name: - -`if_stmt: "if" if_condition ":" body elifs "else" ":" else_body` - -`elif_: "elif" elif_condition ":" elif_body` - -`if_condition` and all `elif_condition`s are resolved in the value name group. - -## Semantics - -If `if_condition` or any `elif_condition` cannot be [coerced](#type-coercion) to [`bool`](#boolean-type), an error is raised. - -At execution, the conditions will be evaluated one at a time until one evaluates to `True`, starting from `if_condition` and going in order through the `elif_conditions`. - -The body of the first condition to evaluate to `True` is executed, and then execution continues after the if statement. - -If no condition evaluates to `True`, and an `else_body` was provided, that body is executed, and then execution continues after the if statement. - -# Loops - -A **loop** executes a block of code zero or more times. - -Each loop has a **loop condition**, which is a Boolean expression which, when `True`, allows the loop body to execute. - -The **enclosing loop** is the loop whose body some source code is in. - -The list of loops is: -* [While loops](#while-loop-statement) -* [For loops](#for-loops) - -## While loop statement - -A **while statement** executes a block of code in a loop while a condition holds `True`. - -### Syntax - -Rule: - -`while_stmt: "while" expr ":" stmt_list` - -Name: - -`while_stmt: "while" condition ":" body` - -`condition` is resolved in the value name group. - -### Semantics - -If `condition` cannot be [coerced](#type-coercion) to [`bool`](#boolean-type), an error is raised. - -The loop condition of a while loop is the provided `condition`. - -At execution: -1. The loop condition is evaluated. -2. If the loop condition is `True`, execute the body, and return to step 1. -3. Otherwise, execution continues after the while loop statement. - -## For loop statement - -A **for loop statement** executes a block of code until a counter reaches an upper bound. - -### Syntax -Rule: - -`for_stmt: "for" name "in" expr ":" stmt_list` - -Name: - -`for_stmt: "for" loop_var "in" range ":" body` - -`loop_var` and `range` are resolved in the value name group. - -### Semantics - -The **loop variable** of a for loop is the variable named by `loop_var`. - -If `loop_var` resolves to a previously-defined variable: -1. If the type of that variable is not [loop var type](#type-aliases), an error is raised. -2. Otherwise, that variable becomes the loop variable of this for loop. - -> This allows reusing the same loop variable name across multiple for loops. - -If `loop_var` does not resolve to a previously-defined variable, a new variable with name `loop_var` and [loop var type](#type-aliases) is added to the [resolving scope](#scopes), and it becomes the loop variable of this for loop. - -> Nothing prevents you from modifying the loop variable in the loop body. However, this may cause infinite loops, so do this with caution. - -If `range` cannot be [coerced](#type-coercion) to [Range type](#internal-types), an error is raised. - -The loop condition of a for loop is `loop_var < upper_bound`, where `upper_bound` is the upper bound of the `range` expression. - -At execution: -1. `range` is evaluated. -2. The loop variable is set to the lower bound of `range`. -3. The loop condition is evaluated. -4. If the loop condition is `True`, execute the body, increment the value of the `loop_var` by 1, and return to step 1. -5. Otherwise, execution continues after the for loop statement. - -> The only possible step size is 1. - -## Break statement - -A **break statement** stops execution of a loop. - -### Syntax -Rule: - -`break_stmt: "break"` - -### Semantics - -If the break statement is outside of a loop body, an error is raised. - -At execution, the enclosing loop body stops executing, and execution is continued after the enclosing loop. - -## Continue statement - -A **continue statement** immediately skips the rest of the loop body, continuing on to the next iteration or ending the loop. - -### Syntax -Rule: - -`continue_stmt: "continue"` - -### Semantics - -If the continue statement is outside of a loop body, an error is raised. - -At execution: -1. The enclosing loop body stops executing. -2. If the loop is a [for loop](#for-loop-statement), the loop variable is incremented. -3. The loop condition is re-evaluated. -4. The loop continues as specified based on the result of the loop condition, either running the body or ending the loop. - -# Assert statement - -An **assert statement** evaluates a Boolean expression and halts the program if the expression evaluates to `False`. - -## Syntax -Rule: - -`assert_stmt: "assert" expr ["," expr]` - -Name: - -`assert_stmt: "assert" condition "," exit_code` - -`condition` and `exit_code` are resolved in the value name group. - -## Semantics - -If `condition` cannot be coerced to [`bool`](#boolean-type), an error is raised. - -If `exit_code` is provided, and cannot be coerced to [`U8`](#primitive-numeric-types), an error is raised. - -At execution, if `condition` evaluates to `False`: -1. If `exit_code` is provided, evaluate it and display its value to the user. -2. If `exit_code` is not provided, display a generic error code to the user. -3. Halt the program. - -# Check statement -The **check statement** executes a block of code if a Boolean expression evaluates to `True` for a duration of time, checking with a configurable frequency and timing out at a configurable time. - -## Syntax -Rule: - -`check_stmt: "check" expr check_clause* ":" stmt_list ["timeout" ":" stmt_list]` - -`check_clause: "timeout" expr | "persist" expr | "freq" expr` - -The clauses can appear in any order, and can be spread across multiple indented lines (with the colon after the last clause). - -Name: - -`check_stmt: "check" condition "timeout" timeout "persist" persist "freq" freq ":" body "timeout" ":" timeout_body` - -`condition`, `timeout`, `persist`, and `freq` are resolved in the value name group. - -## Semantics - -If `condition` cannot be [coerced](#type-coercion) to [`bool`](#boolean-type), an error is raised. - -If `timeout` is provided, and cannot be coerced to [`Fw.Time`](todo), an error is raised. - -If `persist` or `freq` is provided, and they cannot be coerced to [`Fw.TimeIntervalValue`](todo), an error is raised. - -At execution: -1. If provided, `timeout`, `persist` and `freq` are evaluated and stored. -2. If `persist` is not provided, its stored value is a zero-duration `Fw.TimeIntervalValue`. -3. If `freq` is not provided, its stored value is a one-second `Fw.TimeIntervalValue`. -4. If `timeout` was provided and the current time is [greater](todo) than `timeout`'s stored value, the check times out. -5. Evaluate `condition`. -6. If `condition` has evaluated to `True` for duration greater than or equal to `persist`'s stored value, execute `body`, then continue execution after the check statement. -7. Otherwise, [sleep](todo) for `freq`'s stored duration. -8. Go to step 4. - -If the check times out during execution: -1. If `timeout_body` is provided, execute it. -2. Execution continues after the check statement. - -> Not providing `persist`, or providing a zero-duration `persist`, means the `condition` only needs to evaluate to `True` once. -> The timeout defaults to never, and the frequency defaults to once per second. - -If at any point during execution, two times which are [incomparable](todo) are attempted to be compared, the check statement will halt the program as if by an [assertion](#assert-statement), and display an error code. - -# Imports - -> **Note:** The import statement is not yet implemented. - -An **import statement** compiles another Fpy source file, called a **module**, and makes the module's [definitions](#definitions) available in the importing file under a [namespace](#namespaces). - -## Syntax - -Rule: - -`import_stmt: "import" name ("." name)*` - -Name: - -`import_stmt: "import" module_path` - -The **module path** is the entire dotted sequence of names. The first name of a module path is its **root segment**, and the last is its **leaf segment**. - -An import statement is only valid outside an indentation block. - -*Tests:* [1](test/fpy/test_imports.py#L535 "test/fpy/test_imports.py::TestImportOnlyAtTopLevel::test_import_inside_if_block_fails"), [2](test/fpy/test_imports.py#L552 "test/fpy/test_imports.py::TestImportOnlyAtTopLevel::test_import_inside_function_fails") - -## Module resolution - -The **import search path** is an ordered list of directories provided by the environment in which the compiler is invoked. - -> In the command-line compiler, the import search path is the directory containing the file being compiled, followed by each directory passed with `-i`/`--include`, in order. - -A module path `s_0.s_1. ... .s_n` **resolves** in a directory `dir` if the file `dir/s_0/s_1/.../s_n.fpy` exists. - -*Tests:* [1](test/fpy/test_imports.py#L705 "test/fpy/test_imports.py::TestImportDottedPaths::test_single_dotted_import"), [2](test/fpy/test_imports.py#L725 "test/fpy/test_imports.py::TestImportDottedPaths::test_deeply_nested_dotted_import"), [3](test/fpy/test_imports.py#L801 "test/fpy/test_imports.py::TestImportPackagePrecedence::test_module_file_beats_namespace_directory"), [4](test/fpy/test_imports.py#L815 "test/fpy/test_imports.py::TestImportPackagePrecedence::test_package_dir_used_for_dotted_descent") - -The module path of an import statement is resolved in each directory of the import search path, in order. The file from the first directory in which it resolves is the imported module. - -*Tests:* [1](test/fpy/test_imports.py#L856 "test/fpy/test_imports.py::TestImportSearchDirs::test_module_found_in_later_search_dir"), [2](test/fpy/test_imports.py#L871 "test/fpy/test_imports.py::TestImportSearchDirs::test_first_search_dir_shadows_later"), [3](test/fpy/test_imports.py#L887 "test/fpy/test_imports.py::TestImportSearchDirs::test_search_order_respects_dir_order"), [4](test/fpy/test_imports.py#L903 "test/fpy/test_imports.py::TestImportSearchDirs::test_dotted_module_resolved_across_search_dirs") - -If the module path resolves in no directory of the import search path, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L246 "test/fpy/test_imports.py::TestImportErrors::test_missing_module_is_an_error"), [2](test/fpy/test_imports.py#L919 "test/fpy/test_imports.py::TestImportSearchDirs::test_no_search_dirs_cannot_resolve"), [3](test/fpy/test_imports.py#L763 "test/fpy/test_imports.py::TestImportDottedPaths::test_missing_leaf_in_existing_package_is_error"), [4](test/fpy/test_imports.py#L828 "test/fpy/test_imports.py::TestImportPackagePrecedence::test_bare_package_import_is_error"), [5](test/fpy/test_imports.py#L841 "test/fpy/test_imports.py::TestImportPackagePrecedence::test_dotted_leaf_package_import_is_error"), [6](test/fpy/test_imports.py#L297 "test/fpy/test_imports.py::TestImportFileErrors::test_import_path_is_a_directory_fails") - -> Every segment except the leaf names a plain directory; no `__init__.fpy`-style marker file is required. Because only the leaf's `.fpy` file satisfies an import, a file `foo.fpy` always takes precedence over a sibling directory `foo/` for `import foo`, while `import foo.bar` descends into the directory `foo/` regardless of whether `foo.fpy` exists. Importing a name that resolves only to a directory is an error: a directory has no code to import. - -## Semantics - -If the imported module fails to parse or compile, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L279 "test/fpy/test_imports.py::TestImportFileErrors::test_parse_error_in_imported_file_fails") - -> The diagnostic should point into the imported file, not at the import statement. - -If the imported module declares one or more [sequence arguments](todo), an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L228 "test/fpy/test_imports.py::TestImportErrors::test_cannot_import_sequence_with_arguments") - -> A `sequence()` directive with no arguments does not prevent a file from being imported. - -*Tests:* [1](test/fpy/test_imports.py#L254 "test/fpy/test_imports.py::TestImportErrors::test_no_arg_sequence_is_importable") - -An imported module may itself contain import statements. The semantics of this section apply to them recursively, with the module in the role of the importing file. All import statements in a compilation resolve against the same import search path. - -*Tests:* [1](test/fpy/test_imports.py#L574 "test/fpy/test_imports.py::TestImportTransitive::test_transitive_import_works") - -If a module transitively imports itself, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L635 "test/fpy/test_imports.py::TestImportCycles::test_self_import_is_cycle_error"), [2](test/fpy/test_imports.py#L656 "test/fpy/test_imports.py::TestImportCycles::test_mutual_import_is_cycle_error"), [3](test/fpy/test_imports.py#L687 "test/fpy/test_imports.py::TestImportCycles::test_three_way_cycle_error") - -The import statement introduces the module path as a chain of namespaces in the global scope. Each [definition](#definitions) at the top level of the module adds its symbol to the leaf namespace: a symbol `x` defined in module `a.b.c` is named `a.b.c.x` in the importing file, and is not accessible under any shorter name. - -*Tests:* [1](test/fpy/test_imports.py#L85 "test/fpy/test_imports.py::TestImportInlining::test_call_imported_function"), [2](test/fpy/test_imports.py#L123 "test/fpy/test_imports.py::TestImportInlining::test_local_and_imported_names_coexist"), [3](test/fpy/test_imports.py#L324 "test/fpy/test_imports.py::TestImportNamespaceIsolation::test_imported_symbol_requires_module_prefix"), [4](test/fpy/test_imports.py#L362 "test/fpy/test_imports.py::TestImportNamespaceIsolation::test_same_function_name_in_two_modules_no_collision"), [5](test/fpy/test_imports.py#L743 "test/fpy/test_imports.py::TestImportDottedPaths::test_dotted_symbol_requires_full_path") - -Per the [name group](#name-groups) rules, these namespaces exist only in the name groups of the symbols they contain. - -*Tests:* [1](test/fpy/test_imports.py#L461 "test/fpy/test_imports.py::TestImportNameCollisions::test_import_coexists_with_local_variable") - -If a name introduced by an import statement is already mapped, in the same name group and the same scope or namespace, to anything other than a namespace introduced by another import statement, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L421 "test/fpy/test_imports.py::TestImportNameCollisions::test_import_collides_with_local_function"), [2](test/fpy/test_imports.py#L442 "test/fpy/test_imports.py::TestImportNameCollisions::test_import_collides_with_local_variable") - -> Namespaces introduced by imports merge: after `import pkg.a` and `import pkg.b`, the namespace `pkg` contains both `a` and `b`. Because name groups do not intersect, an imported module conflicts only with names in the groups it occupies: a variable `lib` may coexist with an imported module `lib` that defines only functions. - -*Tests:* [1](test/fpy/test_imports.py#L780 "test/fpy/test_imports.py::TestImportDottedPaths::test_two_modules_in_same_package_no_collision") - -Names within the module are resolved in the module's own global scope. Symbols of the importing file are not visible in the module, and modules imported by the module are not visible in the importing file. - -*Tests:* [1](test/fpy/test_imports.py#L392 "test/fpy/test_imports.py::TestImportNamespaceIsolation::test_imported_function_cannot_see_importer_globals"), [2](test/fpy/test_imports.py#L602 "test/fpy/test_imports.py::TestImportTransitive::test_transitive_dependency_is_private") - -If the same module path is imported more than once in the same file, an error is raised. - -*Tests:* [1](test/fpy/test_imports.py#L489 "test/fpy/test_imports.py::TestImportDuplicates::test_duplicate_import_is_error") - -> This rule is per-file: a file and a module it imports may each import the same module. - -*Tests:* [1](test/fpy/test_imports.py#L506 "test/fpy/test_imports.py::TestImportDuplicates::test_duplicate_across_files_is_allowed") - -If the imported module contains top-level statements other than function definitions and import statements, the `import-side-effects` warning is emitted. - -*Tests:* [1](test/fpy/test_imports.py#L158 "test/fpy/test_imports.py::TestImportSideEffects::test_side_effecting_import_warns"), [2](test/fpy/test_imports.py#L172 "test/fpy/test_imports.py::TestImportSideEffects::test_side_effect_warning_can_be_ignored"), [3](test/fpy/test_imports.py#L187 "test/fpy/test_imports.py::TestImportSideEffects::test_side_effect_warning_can_be_escalated"), [4](test/fpy/test_imports.py#L202 "test/fpy/test_imports.py::TestImportSideEffects::test_functions_only_module_does_not_warn"), [5](test/fpy/test_imports.py#L308 "test/fpy/test_imports.py::TestImportFileErrors::test_empty_module_compiles_without_warning") - -At execution, the imported module's top-level statements execute as part of the importing sequence, at the position of the import statement, in order. - -*Tests:* [1](test/fpy/test_imports.py#L106 "test/fpy/test_imports.py::TestImportInlining::test_imported_function_runs"), [2](test/fpy/test_imports.py#L932 "test/fpy/test_imports.py::TestImportVariables::test_top_level_variable_is_side_effect_and_namespaced") - -> Importing is compile-time source inlining: the imported module does not exist in the compiled output, and no files are resolved or loaded at run time. This is what motivates the `import-side-effects` warning: a file written to run as a standalone sequence typically has top-level commands, and importing it splices that code into the importing sequence, which is rarely intended. A file meant to be imported should contain only definitions. Note that a top-level variable definition is such a side effect (its initialization executes at the import site), while still defining a namespaced symbol: `counter: U32 = 5` in module `m` warns, and is thereafter accessible as `m.counter`. - -# Callables - -A **callable** is a symbol with parameters and a return [type](#types) which can be evaluated by being called. - -> Evaluation of a callable always refers to evaluation of a [function call expression](#function-call-expression), where the function is that callable. -> Because callable evaluation is always via a function call expression, when talking about the evaluation of a callable, it is assumed that the evaluation semantics of the function call expression have already occurred. Specifically, the arguments have already been evaluated from left to right. - -Callables can be divided into four categories: -* [Commands](#commands) -* [Functions](#functions) -* [Builtin functions](#builtin-functions) -* [Constructors](todo) - -## Commands -A **command** is a callable with an associated F-Prime command. - -All F-Prime commands in the dictionary have a corresponding Fpy command. - -All Fpy commands are [globally-scoped](#scopes). - -The fully-qualified name of an Fpy command is the same as the corresponding F-Prime command. - -The parameter names and types of an Fpy command are the same as the corresponding F-Prime command. - -> The F-Prime specification requires that all command parameter types be [serializable](#types). - -The return type of all commands is [`Fw.CmdResponse`](todo). - -> Throughout the specification, a "command" means an Fpy command. - -### Command evaluation - -Command evaluation is performed as follows: -1. The argument values are serialized and arranged into the F-Prime command binary format. -2. The binary command is dispatched to the F-Prime system. -3. Execution blocks until the [command response](todo) comes back from the F-Prime system. -4. The expression evaluates to a value of `Fw.CmdResponse` corresponding to that command response. - -### Command responses - -A **command response** is a value of type `Fw.CmdResponse`. - -`Fw.CmdResponse` must be defined in the dictionary, otherwise, an error is raised. -TODO gracefully handle when it's missing. - -## Builtin functions -A **builtin function** is a callable whose behavior is explicit in the specification. - -### `exit` -#### Signature: - -`exit(exit_code: U8)` - -#### Semantics -At evaluation: -1. If `exit_code` evaluates to a non-zero value, display that value to the user. -2. The program is halted. - -### `log` -#### Signature - -`log(operand: F64) -> F64` - -#### Semantics - -At evaluation: -1. If `operand` is outside the domain of the natural logarithm function, halt the program and display an error code. -2. The expression evaluates to the natural logarithm of `operand`. - -### `sleep` -#### Signature - -`sleep(seconds: U32 = 0, useconds: U32 = 0)` - -#### Semantics - -At evaluation, the program [sleeps](#sleeping) for a duration of `seconds` seconds and `useconds` microseconds. - -### `sleep_until` -#### Signature - -`sleep_until(wakeup_time: Fw.Time)` - -#### Semantics - -At evaluation, the program [sleeps](#sleeping) until the given `wakeup_time`. - -### `now` -#### Signature -`now() -> Fw.Time` -#### Semantics - -At evaluation, the function call evaluates to a `Fw.Time` value representing the current time. -TODO this really should be linked to the FpySequencer spec to say exactly where it gets this, etc. we also need engineering details - -### `iabs` -#### Signature -`iabs(value: I64) -> I64` - -#### Semantics -At evaluation, the function call evaluates to the absolute value of `value`. - -TODO specify what happens if the abs value is outside of i64 - -### `fabs` -#### Signature -`fabs(value: F64) -> F64` - -#### Semantics -At evaluation, the function call evaluates to the absolute value of `value`. -TODO specify what happens if the abs value is outside of i64 - -## Builtin libraries - - -## Time functions -TODO Fpy provides builtin functions for comparing and manipulating time values: - -* `time_cmp(lhs: Fw.Time, rhs: Fw.Time) -> I8`: compares two absolute times. Returns `-1` if `lhs` occurs before `rhs`, `0` if they are the same moment, `1` if `lhs` occurs after `rhs`, or `2` if the time bases differ (incomparable). -* `time_interval_cmp(lhs: Fw.TimeIntervalValue, rhs: Fw.TimeIntervalValue) -> I8`: compares two time intervals. Returns `-1` if `lhs` is a shorter duration than `rhs`, `0` if they are the same duration, or `1` if `lhs` is a longer duration than `rhs`. -* `time_sub(lhs: Fw.Time, rhs: Fw.Time) -> Fw.TimeIntervalValue`: subtracts two absolute times, producing a time interval. Asserts that both times have the same time base and that `lhs` occurs after `rhs` (no negative intervals). -* `time_add(lhs: Fw.Time, rhs: Fw.TimeIntervalValue) -> Fw.Time`: adds a time interval to an absolute time, producing a new absolute time. Asserts that the result does not overflow. - -These functions are implemented in Fpy itself (see `src/fpy/builtin/time.fpy`) and are automatically available in all sequences. - -## Constructors -TODO Structs, arrays, and `Fw.Time` expose constructors whose callable name is the fully qualified type name. Their arguments correspond to the members in definition order (struct fields by name, array elements as `e0`, `e1`, ..., and `Fw.Time` with `time_base`, `time_context`, `seconds`, `useconds`). A constructor call serializes the provided values into a new instance of that type. - -## Numeric casts -TODO Each concrete numeric type provides a callable whose name matches the type (for example `U16(value)` or `F64(value)`). Casts accept exactly one numeric argument. Unlike implicit coercion, casts always force the operand into the target type even when this requires narrowing; range checks are suppressed and the value is truncated or rounded if necessary. See [Casting](#casting) for details. - -# Types - -A **type** is a set of **values**. - -The values of a type are unique to that type. - -> In other words, there are no union types and there is no type inheritance. - -New types cannot be defined by the program. - -A **serializable type** is a type whose values can be expressed in a binary format. - -A **constant-sized type** is a serializable type whose binary form always has the same length in bytes. - -A **numeric type** is a [primitive numeric type](#primitive-numeric-types), or the [internal Int or Float](#internal-types) types. - -> Right now, the only serializable but non-constant-sized type are the [dictionary string](#dictionary-strings) types. - -Types can be divided into three categories: -* Primitive types -* Internal types -* Dictionary types - -## Primitive types - -**Primitive types** are types which are always present in the global scope. - -> That is, they do not have to be in the F-Prime dictionary to be referenced by name in the program. -> Because they are present in the global scope, we will use their associated name in the global scope to refer to them throughout this specification. For instance, when we say type `U16`, we are talking about the type in the global scope with name `U16`. - -All primitive types are serializable, constant-sized types. - -The list of primitive types is: -* All [primitive numeric types](#primitive-numeric-types) -* The [Boolean type](#boolean-type) - -### Primitive numeric types -`U8`, `U16`, `U32`, and `U64` are the primitive unsigned integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard binary representation of unsigned integers. - -`I8`, `I16`, `I32`, and `I64` are the primitive signed integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard two's complement representation of signed integers. - -`F32`, and `F64` are the primitive IEEE floating-point types with bitwidths 32 and 64, respectively. - -> There are other numerical types such as [Int or Float](#internal-types) which are not primitive. - -### Boolean type -`bool` is a primitive type whose only values may be the [Boolean literals](todo) `True` and `False`. - -TODO make sure that Fw.Time is counted as a dictionary type, but one which is required to be in the dict? - -## Type aliases -**Loop var type** is an alias for `I64`. -**Array index type** is an alias for `I64`. - -## Internal types - -**Internal types** are types which are never present in the global scope. - -> That is, they cannot be referenced by name in the program. - -No internal types are serializable types. - -**Int** is an internal type whose values are integers of arbitrary precision. - -**Float** is an internal type whose values are decimals per the Python [decimal](https://docs.python.org/3/library/decimal.html#module-decimal) implementation. - -The precision of Float is 30 decimal places. - -**String** is an internal type whose values are strings of arbitrary length. - -**Range** is an internal type whose values are pairs of an lower and upper bound of loop var type. - -**Nothing** is an internal type which has no values. - -## Dictionary types -**Dictionary types** are types defined in the F-Prime dictionary. - -> Because the semantics of these types is defined in the FPP specification, there is some overlap here. This specification just addresses the semantics of these types as relevant to Fpy. - -All dictionary types are serializable types. - -Dictionary types can be divided into three categories: -* [Structs](#structs) -* [Arrays](#arrays) -* [Enums](#enums) -* [Strings](#dictionary-strings) - -### Structs -A **struct** is a category of dictionary type defined by an ordered list of members. - -A **member** is a pair of a name and a serializable type. - -A struct may not have two members with the same name. -TODO is this rule necessary? This is enforced upstream by FPP - -The binary form of a struct value is the concatenated binary forms of its member values, in order. - -> If any of a struct's members are non-constant-sized types, the struct is a non-constant-sized type. - -### Arrays - -An **array** is a category of dictionary type defined by a non-negative integer length, and an element type. - -The **element type** of an array type is the type of its elements. - -An **element** is a value of element type at an index in an array. - -The binary form of an array value is the concatenated binary form of its elements, in order. - -> If the element type is a non-constant-sized type, the array is a non-constant-sized type. - -### Enums - -An **enum** is a category of dictionary type whose values are a finite set of enum constants. - -An **enum constant** is a pair of a name and a value of the enum's representation type. - -An **enum representation type** is the [primitive integer type](#primitive-numeric-types) associated with the enum constants' values. - -The binary form of an enum constant is the binary form of its integer value. - -### Dictionary strings - -A **dictionary string** is a category of dictionary type whose values are strings. - -Dictionary strings are non-constant-sized types. - -## Fields - -A **field-based type** is a type defined by its fields. - -A **field** of a type is a name-and-type pair - -An **array** is a category of type with - -A **struct** is a category of type - -is an [array element](#arrays) or a [struct member](#structs). - -The **field base** of a field is the first non-field parent of a field. - -> For instance, the field base of `a.b.c`, if `a` were a variable and `b` and `c` were fields, would be `a`. - -## Populating dictionary types - -For each type `T` with fully-qualified name `F.Q.N` encountered in the F-Prime dictionary, that type will be present in the global scope with the same fully-qualified name. - -## Type conversion - -TODO Type conversion is the process of converting an expression from one type to another. It can either be implicit, in which case it is called coercion, or explicit, in which case it is called casting. - -### Intermediate types - -The **intermediate type** of a binary or unary operator expression is the type to which all argument expressions will be coerced to. - -Intermediate types are picked via the following rules: - -1. The intermediate type of Boolean operators is always `bool`. -2. The intermediate type of `==` and `!=` may be any type, so long as the left and right hand sides are the same type. If both are numeric then continue. -3. If either argument is non-numeric, raise an error. -4. If the operator is `/` or `**`, the intermediate type is always `F64`. -5. If either argument is a float, the intermediate type is `F64`. -6. If either argument is an unsigned integer, the intermediate type is `U64`. -7. Otherwise, the intermediate type is `I64`. - -If the expressions given to the operator are not of the intermediate type, type coercion rules are applied. - -## Result type - -The result type is the type of the value produced by the operator. -1. For numeric operators, the result type is the intermediate type. -2. For boolean and comparison operators, the result type is `bool`. - -Normal type coercion rules apply to the result, of course. Once the operator has produced a value, it may be coerced into some other type depending on context. - -# Expressions - -An **expression** can be evaluated to produce a value of a type. - -A **constant expression** is an expression which can be evaluated without running the program. - -## Literals - -A **literal** is an expression whose value is explicit in the source code. - -All literal expressions are constant expressions. - -### Integer literals - -#### Decimal literal syntax - -Rule: - -``` -DEC_LITERAL: "1".."9" ("_"? "0".."9")* - | "0" ("_"? "0")* /(?![1-9xX])/ -``` - -#### Hexadecimal literal syntax - -Rule: - -`HEX_LITERAL: ("0x" | "0X") ("_"? /[0-9a-fA-F]/)+` - -#### Semantics - -Integer literals have type [Int](#internal-types). - -### Float literals - -#### Syntax -``` -_SPECIAL_DEC: "0".."9" ("_"? "0".."9")* - -DECIMAL: "." _SPECIAL_DEC | _SPECIAL_DEC "." _SPECIAL_DEC -_EXP: ("e"|"E") ["+" | "-"] _SPECIAL_DEC -FLOAT_LITERAL: _SPECIAL_DEC _EXP | DECIMAL _EXP? -``` - -Float literals have type [Float](#internal-types). - -A float literal is rounded to the nearest value of type Float. - -### String literals -#### Syntax - -Rule: - -`STRING_LITERAL: /("(?!"").*?(? Namespaces, types names, and function names are valid expressions syntactically, but not semantically. Thus, trying to access a member of either of these symbols will raise an error. - -If the type of `parent` is not a [struct](#structs), an error is raised. - -If the type of `parent` is not [constant-sized](#types), an error is raised. - -If `member` is not a member of the type of `parent`, an error is raised. - -The type of a member access is the type of the `member` in the type of `parent`. - -At evaluation: -1. The `parent` is evaluated. -2. The member access expression evaluates to the value of the `member` in the `parent` value. - -## Function call expression -### Syntax - -Rule: - -``` -func_call: expr "(" [arguments] ")"` -arguments: argument ("," argument)* -argument: NAME "=" expr -> named_argument - | expr -> positional_argument -``` - -Name: - -`func_call: func "(" arguments ")"` - -`func` is resolved in the callable name group. - -All argument expressions are resolved in the value name group. - -### Semantics - -At evaluation: -1. Each argument is evaluated from left to right. -2. The behavior defined in the semantics for the [callable](#callables) referenced by `func` is - -## Binary operator expressions - -A **binary operator expression** is an expression with a left and right-hand expression, and a binary operator in between, which acts on both values to produce a new value. - -The list of **binary operators** is: -* The [addition operator](#subtraction-semantics) `+` -* The [subtraction operator](#multiplication-semantics) `-` -* The [multiplication operator](#multiplication-semantics) `*` -* The [division operator](#division-semantics) `/` -* The [floor division operator](#floor-division-semantics) `//` -* The [modulus operator](#modulus-semantics) `%` -* The [exponentiation operator](#exponentiation-semantics) `**` -* The [Boolean operators](#boolean-operator-semantics) `and` and `or` -* The [comparison operators](#comparison-semantics) `>`, `>=`, `<`, and `<=` -* The [equality operator](#equality-semantics) `==` -* The [inequality operator](#inequality-semantics) `!=` -* The [range operator](#range-semantics) `..` - -### Syntax - -Rule: - -`binary_op: expr BINARY_OP expr` - -Name: - -`binary_op: lhs op rhs` - -`lhs` and `rhs` are resolved in the value name group. - -### Semantics - -For each use of a binary operator, an [intermediate type](#intermediate-types) is picked, as described in the operator's semantics. - -If `lhs` or `rhs` cannot be [coerced](#type-coercion) into the intermediate type, an error is raised. - -If `lhs` and `rhs` are constant expressions, the binary operator expression is a constant expression. - -At evaluation, for all operators besides the [Boolean operators](#boolean-operator-semantics): -1. `lhs` is evaluated and coerced into the intermediate type. -2. `rhs` is evaluated and coerced into the intermediate type. -3. The expression evaluates to a value of the intermediate type, as described in the operator's semantics. - -#### Addition semantics -The addition operator is `+`. - -If neither `lhs` nor `rhs` are expressions of a [numeric type](#types), an error is raised. - -The expression evaluates to the result of adding - -#### Subtraction semantics -#### Multiplication semantics - -These operators require numeric operands and produce a result in the chosen intermediate type. Addition, subtraction, and multiplication differ only in which arithmetic operation they perform. Integer overflow wraps according to the destination type when the result is ultimately stored, and floating-point operations follow IEEE-754 behavior. - -#### Division semantics -Both operands are promoted to `F64`, and the result is always an `F64`. This means you must explicitly cast the result to store it in an integer type. - -#### Floor division semantics -With integer operands, `//` performs truncating division using the signed or unsigned divide directive. If either operand is a float, the compiler divides in `F64`, converts the quotient to a signed 64-bit integer (which truncates toward zero), and converts back to `F64`, so floating-point floor division also truncates toward zero. - -### Modulus semantics -Modulus works for numeric operands. Signed operands use the signed modulo directive, unsigned operands use the unsigned directive, and floats use floating-point modulo. For signed integers the remainder has the same sign as the dividend. - -#### Exponentiation semantics -Both operands are coerced to `F64`, the exponentiation happens in floating point, and the result type is `F64`. - -#### Boolean operator semantics -Operands must be `bool`. `not` negates a single operand. `and` evaluates the left operand first and only evaluates the right operand when the left operand is `True`. Conversely, `or` skips the right operand when the left operand is `True`. The result of every boolean operator is `bool`. - -#### Comparison semantics -Inequalities require numeric operands. Each operand is coerced to the intermediate type, the comparison runs in that type, and the result is `bool`. - -#### Equality semantics -If both operands are numeric, equality uses the same intermediate-type rules as arithmetic operators. Otherwise both operands must have the exact same concrete type (struct, array, enum, or `Fw.Time`). The compiler compares their serialized bytes. Strings cannot be compared. - -#### Range semantics -The range operator is `..`. - -If `lhs` or `rhs` cannot be coerced to [loop var type](#type-aliases), an error is raised. - -#### Order of operations -The order in which operations take precedence, from most strongly binding to least strongly binding, is: -1. [Exponentiation](#exponentiation-semantics) -2. [Negation](#negation-operator-semantics) and [identity](#identity-operator-semantics) -3. [Multiplication](#multiplication-semantics), [division](#division-semantics), [floor division](#floor-division-semantics), and [modulus](#modulus-semantics) -4. [Addition](#addition-semantics) and [subtraction](#subtraction-semantics) -5. [Range](#range-semantics) -6. [Comparison](#comparison-semantics) -7. [Not](#boolean-operator-semantics) -8. [And](#boolean-operator-semantics) -9. [Or](#boolean-operator-semantics) - -If two operators have the same precedence in the above list, then the leftmost operator binds more strongly. - -## Unary operators -### Syntax - -Rule: - -`unary_op: expr OP` - -Name: - -`unary_op: val op` - -### Negation operator semantics -### Identity operator semantics - -## Intermediate types - -The **intermediate type** of an operator expression is the type to which the operator's sub-expressions are [coerced](#type-coercion) to. - -If any sub-expression - -### Numeric intermediate types - -The numeric type hierarchy is as follows: -* - - - - # we split this algo up into two stages: picking the type category (float, uint or int), and picking the type bitwidth - - # pick the type category: - type_category = None - if op == BinaryStackOp.DIVIDE or op == BinaryStackOp.EXPONENT: - # always do true division and exponentiation over floats, python style - # this is because, for the given op, even with integer inputs, we might get - # float outputs - type_category = "float" - elif any(issubclass(t, FloatValue) for t in arg_types): - # otherwise if any args are floats, use float - type_category = "float" - elif any(t in UNSIGNED_INTEGER_TYPES for t in arg_types): - # otherwise if any args are unsigned, use unsigned - type_category = "uint" - else: - # otherwise use signed int - type_category = "int" - - # pick the bitwidth - # we only use the arb precision types for constants, so if theyre all arb precision, they're consts - constants = all(t in ARBITRARY_PRECISION_TYPES for t in arg_types) - - if constants: - # we can constant fold this, so use infinite bitwidth - if type_category == "float": - return FpyFloatValue - assert type_category == "int" or type_category == "uint" - return FpyIntegerValue - - # can't const fold - if type_category == "float": - return F64Value - if type_category == "uint": - return U64Value - assert type_category == "int" - return I64Value - - -## Type conversion - -**Type conversion** is the process by which values of one type are converted into values of another type. - -There are two kinds of type conversion: -* [Casting](#casting) -* [Coercion](#type-coercion) - -Type casting is merely an explicit flag for type coercion to take place - - -### Type coercion -**Type coercion** is type conversion that happens implicitly to an expression when required by that expression's semantic context. - - -Coercion happens when an expression of type *A* is used in a syntactic element which requires an expression of type *B*. For example, functions, operators and variable assignments all require specific input types, so type coercion happens in each of these. -In general, the rule of thumb is that coercion is allowed if the destination type can represent all possible values of the source type, with some exceptions. The following rules determine when type coercion can be performed: - -1. If the source and destination types are identical, no coercion is performed. -2. *LiteralString* values may be coerced into any FPP string type. No other string expression can be coerced. -3. Otherwise both source and destination must be numeric (`NumericalValue`). Numeric coercions obey these constraints: - * Floats never coerce to integers. - * Integers may always coerce to floats. - * Float-to-float coercions require a destination bit width greater than or equal to the source width. - * Integer-to-integer coercions require matching signedness and a destination bit width greater than or equal to the source width. - * Arbitrary-precision types (`Int`/`Float`) may coerce to any finite-width numeric type. -If no rule matches, the compiler raises an error. - -Compile-time constant floats (including literals and constant-folded expressions) can only be narrowed into a smaller floating-point type when the value lies inside the destination's representable range. When the value fits, the compiler rounds it to the nearest representable floating-point number; otherwise compilation fails with an out-of-range error. - - -# Execution - -## Control flow -A **branch** is a block of code which conditionally executes. - -> That is, whether or not that code executes depends on some expression. - -The list of statements which have branches is: -* The [if statement](#ifs) -* The [while loop statement](#while-loop-statement) -* The [for loop statement](#for-loop-statement) -* The [check statement](#check-statement) -TODO how does this definition handle assert/exit? - -## Sleeping -The program may **sleep** until an absolute time called the **wakeup time**. - -While the current time is before the wakeup time, no statements may execute - -The program may also sleep for a time duration. This is the same as sleeping with a wakeup time of now, plus the specified duration. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..225345c --- /dev/null +++ b/docs/README.md @@ -0,0 +1,48 @@ +# Fpy specification docs + +The specification is written in [AsciiDoc](https://asciidoc.org/), one file per +section, and rendered to a single self-contained HTML page with +[Asciidoctor.js](https://github.com/asciidoctor/asciidoctor.js), run via `npx` +so no Ruby toolchain is needed -- only Node, which the repo already uses. + +The `.adoc` files under `spec/` are the **source of truth** -- edit them +directly. + +## Layout + +- `spec.adoc` -- master document: header, introduction, and `include::` lines + pulling in each section in order. +- `spec/NN-*.adoc` -- one file per top-level section. +- `build.sh` -- renders `spec.adoc` to `spec.html`. + +## Build + +```sh +docs/build.sh +``` + +This writes `docs/spec.html`, a standalone page with styles and a left-hand +table of contents inlined. `spec.html` is a build artifact. + +## Preview / serve + +Open `docs/spec.html` directly, or serve the directory: + +```sh +python3 -m http.server -d docs 8000 # then visit http://localhost:8000/spec.html +``` + +## Editing + +Edit the `spec/*.adoc` files. To add a section, create `spec/NN-name.adoc` and +add an `include::` line to `spec.adoc` -- keep a blank line after each +`include::` so Asciidoctor keeps the sections at the top level. + +Cross references use `<>`, where `slug` is a heading's anchor +(lowercase, spaces to hyphens); the `:idprefix:`/`:idseparator:` settings in +`spec.adoc` make Asciidoctor generate those IDs. + +Each statement may be followed by a `_Tests:_` line linking to the tests that +verify it. Those links are checked by `verify/spec_links.py` (a pre-commit +hook); run `uv run python verify/spec_links.py --fix` to refresh stale line +numbers. diff --git a/docs/build.sh b/docs/build.sh new file mode 100755 index 0000000..1641322 --- /dev/null +++ b/docs/build.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +# Build the Fpy specification into a single self-contained HTML page. +# +# Renders docs/spec.adoc (which include::s the per-section files under +# docs/spec/) with Asciidoctor.js, run through npx so no Ruby toolchain is +# needed -- only Node, which the repo already uses. +# +# Usage: docs/build.sh [output.html] (default: docs/spec.html) +set -euo pipefail + +cd "$(dirname "$0")" +out="${1:-spec.html}" + +npx --yes asciidoctor -o "$out" spec.adoc + +echo "Built docs/$out" diff --git a/docs/spec.adoc b/docs/spec.adoc new file mode 100644 index 0000000..f86d3ed --- /dev/null +++ b/docs/spec.adoc @@ -0,0 +1,59 @@ += Fpy Specification +:toc: left +:toclevels: 3 +:sectnums: +:sectlinks: +:sectanchors: +:idprefix: +:idseparator: - +:source-highlighter: highlight.js + + +Fpy is a sequencing language for the F-Prime flight software framework. It combines Python-like syntax with the FPP model, with domain-specific features for spacecraft operation. + +This document specifies the syntax and semantics of the Fpy sequencing language. In other words, it specifies which programs the compiler should accept, and how the output should behave when run. The intent is to leave the implementation of the compiler, the bytecode it generates, and the virtual machine it runs on, unspecified. + +It is assumed the reader is familiar with the FPP model, including commands, telemetry, parameters, structs, arrays and enums. + +> Informal notes and explanations are quoted like this. + +Terms are *bolded* in the location they are primarily defined. + +`Monospaced` text refer to syntactic rules, type names or example Fpy code. + +In a syntactic rule: + +* Text in between forward slashes `/` is regex +* Text in between square brackets `[]` is optional +* Text in between parentheses `()` is handled as a group +* A plus suffix `+` means one or more instances of its preceding rule +* A star suffix `*` means zero or more instances of its preceding rule +* A question mark suffix `?` means zero or one instances of its preceding rule + +A line of the specification may be followed by an italicized _Tests:_ line. Each bracketed number links to a test that verifies the behavior; hover over a number to see the test's full `file::class::name`. These links are checked and kept up to date by `verify/spec_links.py`. + + + +include::spec/01-names-and-scopes.adoc[] + +include::spec/02-variables.adoc[] + +include::spec/03-functions.adoc[] + +include::spec/04-ifs.adoc[] + +include::spec/05-loops.adoc[] + +include::spec/06-assert-statement.adoc[] + +include::spec/07-check-statement.adoc[] + +include::spec/08-imports.adoc[] + +include::spec/09-callables.adoc[] + +include::spec/10-types.adoc[] + +include::spec/11-expressions.adoc[] + +include::spec/12-execution.adoc[] diff --git a/docs/spec.html b/docs/spec.html new file mode 100644 index 0000000..5feecd0 --- /dev/null +++ b/docs/spec.html @@ -0,0 +1,2854 @@ + + + + + + + +Fpy Specification + + + + + + +
+
+
+
+

Fpy is a sequencing language for the F-Prime flight software framework. It combines Python-like syntax with the FPP model, with domain-specific features for spacecraft operation.

+
+
+

This document specifies the syntax and semantics of the Fpy sequencing language. In other words, it specifies which programs the compiler should accept, and how the output should behave when run. The intent is to leave the implementation of the compiler, the bytecode it generates, and the virtual machine it runs on, unspecified.

+
+
+

It is assumed the reader is familiar with the FPP model, including commands, telemetry, parameters, structs, arrays and enums.

+
+
+
+
+

Informal notes and explanations are quoted like this.

+
+
+
+
+

Terms are bolded in the location they are primarily defined.

+
+
+

Monospaced text refer to syntactic rules, type names or example Fpy code.

+
+
+

In a syntactic rule:

+
+
+
    +
  • +

    Text in between forward slashes / is regex

    +
    • +
  • +

    Text in between square brackets [] is optional

    +
    • +
  • +

    Text in between parentheses () is handled as a group

    +
    • +
  • +

    A plus suffix + means one or more instances of its preceding rule

    +
    • +
  • +

    A star suffix * means zero or more instances of its preceding rule

    +
    • +
  • +

    A question mark suffix ? means zero or one instances of its preceding rule

    +
    • +
+
+
+

A line of the specification may be followed by an italicized Tests: line. Each bracketed number links to a test that verifies the behavior; hover over a number to see the test’s full file::class::name. These links are checked and kept up to date by verify/spec_links.py.

+
+
+
+
+

1. Names and scopes

+
+
+

1.1. Names

+
+

name: /\$?[^\W\d]\w*/

+
+
+

A name is a string consisting of letters, underscores or digits. The first character may not be a digit.

+
+
+

The first character may optionally be $, in which case the name is considered escaped. An escaped name is the same as an unescaped name, except in that it always lexes as a name, even if it is a reserved word.

+
+
+
+

1.2. Reserved words

+
+

A reserved word is a word which cannot be used as an unescaped name.

+
+
+

The list of reserved words is:

+
+
+
    +
  • +

    assert

    +
    • +
  • +

    break

    +
    • +
  • +

    check

    +
    • +
  • +

    continue

    +
    • +
  • +

    def

    +
    • +
  • +

    for

    +
    • +
  • +

    if

    +
    • +
  • +

    import

    +
    • +
  • +

    not

    +
    • +
  • +

    pass

    +
    • +
  • +

    return

    +
    • +
  • +

    while

    +
    • +
+
+
+
+

1.3. Symbols

+
+

A symbol is a language construct that can be referred to by a name in the program.

+
+
+

The following language constructs may be symbols:

+
+
+
    +
  • +

    namespaces

    +
    • +
  • +

    variables

    +
    • +
  • +

    callables

    +
    • +
  • +

    types

    +
    • +
  • +

    telemetry channels

    +
    • +
  • +

    parameters

    +
    • +
  • +

    enum constants +TODO members? +TODO you’re selecting a member of a value—​not referring to a static member. when you say a.b you’re asking for a comp to be performed

    +
    • +
+
+
+
+

1.4. Scopes

+
+

A scope is a mapping of names to symbols, accessed via some region of the source code. +TODO a scope IS a REGION

+
+
+

The global scope is the scope accessible throughout the entire source code.

+
+
+

Each function has a function scope, accessible in its body.

+
+
+

The resolving scope is the most specific scope that some part of the source code has access to.

+
+
+

Scopes may have a parent scope:

+
+
+
    +
  • +

    The parent scope of a function scope is the global scope.

    +
    • +
  • +

    The global scope does not have a parent scope.

    +
    • +
+
+
+
+

1.5. Name groups

+
+

Scopes are divided into name groups.

+
+
+

The list of name groups is:

+
+
+ +
+
+

Each name group only contains names which map to their particular language construct, or namespaces with the same property, recursively. +TODO Maybe could remove this line

+
+
+

TODO explain why we need this +Name groups do not intersect.

+
+
+
+
+

This means that the names of callables, types and values never conflict. WITH EACH OTHER

+
+
+
+
+

Name groups are accessed via syntactic context. +TODO this is really just an explanation

+
+
+

TODO type names should NOT be expressions

+
+
+

TODO A.b is an expr, A is an expr, b is not an expression. A.b is a dot expression. b is not an expression, it’s part of a +TODO Does it refer to something that has a scope, or does it refer to something that has members.

+
+
+
+
+

For instance, the type name group is accessible anywhere in the source code where a type name is expected, such as a variable definition type annotation, or a function definition return type.

+
+
+
+
+

The resolving name group is the name group that a name should be resolved in, based on its syntactic context.

+
+
+
+

1.6. Namespaces

+
+

A namespace is a mapping of names to symbols, associated with a name.

+
+
+
+

1.7. Qualified names

+
+

A qualified name is one of:

+
+
+
    +
  • +

    A name

    +
    • +
  • +

    A qualified name, followed by a ., followed by a name

    +
    • +
+
+
+

The qualifier is the qualified name to the left of the ..

+
+
+

To resolve a qualified name in a name group:

+
+
+
    +
  1. +

    If there is no qualifier:

    +
    +
      +
    1. +

      Resolve the name in the resolving scope.

      +
      2. +
    2. +

      If the name fails to be resolved, resolve the name in the parent scope of the resolving scope.

      +
      3. +
    +
    +
    2. +
  2. +

    Otherwise:

    +
    +
      +
    1. +

      Resolve the qualifier.

      +
      2. +
    2. +

      If the qualifier is an expression, resolution is handled by the rules of member access.

      +
      3. +
    3. +

      If the qualifier is not a namespace, an error is raised.

      +
      4. +
    4. +

      Resolve the name in the qualifier namespace.

      +
      5. +
    +
    +
    3. +
+
+
+

If at any point a name fails to be resolved, an error is raised, unless otherwise specified.

+
+
+

A fully-qualified name is a qualified name which is not itself a qualifier. +TODO names are semantic, ident is syntactic +TODO you can’t actually tell at syntax level what is a fqn +If a fully-qualified name resolves to a namespace, an error is raised.

+
+
+

Tests: 1

+
+
+

TODO you can think of the dict as "importing definitions"

+
+
+
+
+

Namespace symbols cannot be used anywhere, so this forces names to resolve to something "useful"

+
+
+
+
+

TODO I’m not sure this is clear what this means. The idea here is that the full qualified name should always reference SOMETHING—​cannot just put Svc in place of a type, even though Svc does resolve in type name group and global scope.

+
+
+
+

1.8. Definitions

+
+

A definition is a language construct that introduces a name-to-symbol mapping as apart of a scope and name group.

+
+
+

The list of definitions is:

+
+
+ +
+
+
+
+
+

2. Variables

+
+
+

A variable is a symbol with a static type and a dynamic, mutable value. +todo maybe call it dynamic?

+
+
+

TODO: give a list of statements

+
+
+

2.1. Variable definition

+
+

A variable definition statement introduces a name-to-variable mapping in its resolving scope.

+
+
+

2.1.1. Syntax

+
+

Rule:

+
+
+

variable_declare_stmt: name ":" qualified_name "=" expr

+
+
+

Name:

+
+
+

variable_declare_stmt: lhs ":" type_ann "=" rhs

+
+
+

lhs and rhs are resolved in the value name group.

+
+
+

type_ann is resolved in the type name group.

+
+
+
+

2.1.2. Semantics

+
+

If lhs resolves to a previously-defined symbol, an error is raised.

+
+
+
+
+

This prevents redefining a variable.

+
+
+
+
+

If rhs cannot be coerced to type type_ann, an error is raised.

+
+
+

The new variable has type type_ann. It is added to the resolving scope under name lhs.

+
+
+

At execution, rhs is evaluated and coerced to type type_ann. This becomes the variable’s initial value.

+
+
+

For statements following this, the variable lhs is considered defined.

+
+
+
+
+

2.2. Variable assignment

+
+

A variable assignment statement mutates the value of a variable.

+
+
+

2.2.1. Syntax

+
+

Rule:

+
+
+

variable_assign_stmt: name "=" expr

+
+
+

Name:

+
+
+

variable_assign_stmt: lhs "=" rhs

+
+
+

lhs and rhs are resolved in the value name group.

+
+
+
+

2.2.2. Semantics

+
+

If lhs does not resolve to a variable, an error is raised.

+
+
+
+
+

Hereafter, lhs refers to the variable named lhs.

+
+
+
+
+

If lhs has not been defined yet, an error is raised.

+
+
+

If rhs cannot be coerced to `lhs’s type, an error is raised.

+
+
+

At execution, rhs is evaluated and coerced to `lhs’s type. This becomes the `lhs’s new value.

+
+
+
+
+

2.3. Member assignment

+
+

A member assignment statement mutates the value of a member in a variable.

+
+
+

2.3.1. Syntax

+
+

Rule:

+
+
+

variable_assign_member_stmt: expr "." name "=" expr

+
+
+

Name:

+
+
+

variable_assign_member_stmt: parent "." member "=" rhs

+
+
+

parent, and rhs are resolved in the value name group.

+
+
+
+

2.3.2. Semantics

+
+

If parent is not a variable or a field with a field base that is a variable, an error is raised.

+
+
+
+
+

This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you’re modifying is a variable.

+
+
+
+
+

If `parent’s type is:

+
+
+
    +
  • +

    not constant-sized, or

    +
    • +
  • +

    not a struct type, or

    +
    • +
  • +

    member is not a member of `parent’s type

    +
    +
      +
    1. +

      an error is raised.

      +
      2. +
    +
    +
    • +
+
+
+

If the variable has not been defined yet, an error is raised.

+
+
+

If rhs cannot be coerced to the member’s type, an error is raised.

+
+
+

At execution, rhs is evaluated and coerced to the member’s type. This becomes the member’s new value.

+
+
+

The value of the variable is unchanged except for the member.

+
+
+
+
+

2.4. Element assignment

+
+

An element assignment statement mutates the value of an element in a variable.

+
+
+

2.4.1. Syntax

+
+

Rule:

+
+
+

variable_assign_element_stmt: expr "[" expr "]" "=" expr

+
+
+

Name:

+
+
+

variable_assign_element_stmt: parent "[" item "]" "=" rhs

+
+
+

parent, item and rhs are resolved in the value name group.

+
+
+
+

2.4.2. Semantics

+
+

If parent is not a variable or a field with a field base that is a variable, an error is raised.

+
+
+
+
+

This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you’re modifying is a variable.

+
+
+
+
+

If the variable has not been defined yet, an error is raised.

+
+
+

If item cannot be coerced to array index type, an error is raised.

+
+
+

If `parent’s type is:

+
+
+
    +
  • +

    not constant-sized, or

    +
    • +
  • +

    not an array type, or

    +
    • +
  • +

    item is a constant with a value less than 0 or greater than the `parent’s type length,

    +
    +
      +
    1. +

      an error is raised.

      +
      2. +
    +
    +
    • +
+
+
+

If rhs cannot be coerced to the element’s type, an error is raised.

+
+
+

At execution:

+
+
+
    +
  1. +

    rhs is evaluated and coerced to the element’s type

    +
    2. +
  2. +

    item is evaluated and coerced to array index type

    +
    3. +
  3. +

    If item is less than zero or greater than the `parent’s type length, a runtime error is raised

    +
    4. +
  4. +

    The element in the parent array at the index item is set to the result of step 1

    +
    5. +
+
+
+

The value of the variable is unchanged except for the element.

+
+
+
+
+

2.5. Variable evaluation

+
+

The value produced by evaluating a variable is the value most recently assigned to that variable, or the initial value if it has only been defined.

+
+
+

If a variable is evaluated before it has been defined, an error is raised.

+
+
+
+
+
+

3. Functions

+
+
+

A function is a callable symbol with an inner scope, parameters, code and a return type.

+
+
+

The call site is the location in the source code at which a function is called.

+
+
+

3.1. Function parameters

+
+

A function parameter is a variable implicitly defined by a function in that function’s scope.

+
+
+

When a function is called, each parameter is set to an initial value.

+
+
+

The initial value may either be from a passed argument, or a default value, if one is specified in the function definition.

+
+
+
+
+

In all other respects, parameters are like normal variables, meaning you can modify them in the function body.

+
+
+
+
+
+

3.2. Return types

+
+

The return type of a function is the type of the value returned by that function. If the return type is Nothing, the function does not return a value.

+
+
+
+

3.3. Returns

+
+

A return statement ends the currently executing function, resumes execution at the call site, and optionally returns a value.

+
+
+

3.3.1. Syntax

+
+

Rule:

+
+
+

return_stmt: "return" [expr]

+
+
+

Name:

+
+
+

return_stmt: "return" value

+
+
+

value is resolved in the value name group.

+
+
+
+

3.3.2. Semantics

+
+

If the return statement is outside of a function body, an error is raised.

+
+
+

The enclosing function of a return statement is the function whose body that return is in.

+
+
+

If value is not provided and the enclosing function’s return type is not Nothing, an error is raised.

+
+
+

If value is provided and cannot be coerced to the return type of the enclosing function, an error is raised.

+
+
+

At execution:

+
+
+
    +
  1. +

    If provided, value is evaluated and coerced to the return type of the enclosing function.

    +
    2. +
  2. +

    The execution of the function body is stopped, and execution after the function call site resumes.

    +
    3. +
+
+
+
+
+

3.4. Function definition

+
+

A function definition statement introduces a name-to-function mapping in the global scope.

+
+
+

3.4.1. Syntax

+
+

Rule:

+
+
+
+
function_def_stmt: "def" name "(" [parameters] ")" ["->" qualified_name] ":" block
+parameters: parameter ("," parameter)*
+parameter: name ":" qualified_name ["=" expr]
+
+
+
+

Name:

+
+
+
+
function_def_stmt: "def" name "(" parameters ")" "->" return_type ":" body
+parameters: parameter_0 "," parameter_1 ... "," parameter_n
+parameter: parameter_name ":" parameter_type "=" parameter_default_value
+
+
+
+

A function definition statement is only valid outside an indentation block.

+
+
+

The parameter `name`s are resolved in the value name group.

+
+
+

name is resolved in the callable name group.

+
+
+

return_type and each of the parameter types are resolved in the type name group.

+
+
+
+

3.4.2. Semantics

+
+

If name resolves to a previously-defined callable, an error is raised.

+
+
+

A new function scope is created, accessible to the body and the parameter `name`s.

+
+
+

Each parameter is a variable in this new scope.

+
+
+
+
+

This implies that no two parameters may have the same name, otherwise they would be conflicting variables.

+
+
+
+
+

If the default value of a parameter is not a constant, an error is raised.

+
+
+

If the default value of a parameter cannot be coerced to the type of the parameter, an error is raised.

+
+
+

If a parameter without a default value follows a parameter with a default value, an error is raised.

+
+
+

If return_type is provided, and any branch of the function does not return a value, an error is raised. +TODO need a section on control flow?

+
+
+

The new function with name name is added to the global scope. If return_type is not provided, the return type is Nothing, otherwise the return type is type return_type.

+
+
+
+
+

Because functions can only be defined in the global scope, you cannot declare a function in a function.

+
+
+
+
+
+
+

Functions can be used before they are defined.

+
+
+
+
+
+
+
+
+

4. Ifs

+
+
+

An if statement conditionally executes blocks of code.

+
+
+

4.1. Syntax

+
+

Rule:

+
+
+

if_stmt: "if" expr ":" stmt_list elifs ["else" ":" stmt_list]

+
+
+

elifs: elif_*

+
+
+

elif_: "elif" expr ":" stmt_list

+
+
+

Name:

+
+
+

if_stmt: "if" if_condition ":" body elifs "else" ":" else_body

+
+
+

elif_: "elif" elif_condition ":" elif_body

+
+
+

if_condition and all `elif_condition`s are resolved in the value name group.

+
+
+
+

4.2. Semantics

+
+

If if_condition or any elif_condition cannot be coerced to bool, an error is raised.

+
+
+

At execution, the conditions will be evaluated one at a time until one evaluates to True, starting from if_condition and going in order through the elif_conditions.

+
+
+

The body of the first condition to evaluate to True is executed, and then execution continues after the if statement.

+
+
+

If no condition evaluates to True, and an else_body was provided, that body is executed, and then execution continues after the if statement.

+
+
+
+
+
+

5. Loops

+
+
+

A loop executes a block of code zero or more times.

+
+
+

Each loop has a loop condition, which is a Boolean expression which, when True, allows the loop body to execute.

+
+
+

The enclosing loop is the loop whose body some source code is in.

+
+
+

The list of loops is:

+
+
+ +
+
+

5.1. While loop statement

+
+

A while statement executes a block of code in a loop while a condition holds True.

+
+
+

5.1.1. Syntax

+
+

Rule:

+
+
+

while_stmt: "while" expr ":" stmt_list

+
+
+

Name:

+
+
+

while_stmt: "while" condition ":" body

+
+
+

condition is resolved in the value name group.

+
+
+
+

5.1.2. Semantics

+
+

If condition cannot be coerced to bool, an error is raised.

+
+
+

The loop condition of a while loop is the provided condition.

+
+
+

At execution:

+
+
+
    +
  1. +

    The loop condition is evaluated.

    +
    2. +
  2. +

    If the loop condition is True, execute the body, and return to step 1.

    +
    3. +
  3. +

    Otherwise, execution continues after the while loop statement.

    +
    4. +
+
+
+
+
+

5.2. For loop statement

+
+

A for loop statement executes a block of code until a counter reaches an upper bound.

+
+
+

5.2.1. Syntax

+
+

Rule:

+
+
+

for_stmt: "for" name "in" expr ":" stmt_list

+
+
+

Name:

+
+
+

for_stmt: "for" loop_var "in" range ":" body

+
+
+

loop_var and range are resolved in the value name group.

+
+
+
+

5.2.2. Semantics

+
+

The loop variable of a for loop is the variable named by loop_var.

+
+
+

If loop_var resolves to a previously-defined variable:

+
+
+
    +
  1. +

    If the type of that variable is not loop var type, an error is raised.

    +
    2. +
  2. +

    Otherwise, that variable becomes the loop variable of this for loop.

    +
    3. +
+
+
+
+
+

This allows reusing the same loop variable name across multiple for loops.

+
+
+
+
+

If loop_var does not resolve to a previously-defined variable, a new variable with name loop_var and loop var type is added to the resolving scope, and it becomes the loop variable of this for loop.

+
+
+
+
+

Nothing prevents you from modifying the loop variable in the loop body. However, this may cause infinite loops, so do this with caution.

+
+
+
+
+

If range cannot be coerced to Range type, an error is raised.

+
+
+

The loop condition of a for loop is loop_var < upper_bound, where upper_bound is the upper bound of the range expression.

+
+
+

At execution:

+
+
+
    +
  1. +

    range is evaluated.

    +
    2. +
  2. +

    The loop variable is set to the lower bound of range.

    +
    3. +
  3. +

    The loop condition is evaluated.

    +
    4. +
  4. +

    If the loop condition is True, execute the body, increment the value of the loop_var by 1, and return to step 1.

    +
    5. +
  5. +

    Otherwise, execution continues after the for loop statement.

    +
    6. +
+
+
+
+
+

The only possible step size is 1.

+
+
+
+
+
+
+

5.3. Break statement

+
+

A break statement stops execution of a loop.

+
+
+

5.3.1. Syntax

+
+

Rule:

+
+
+

break_stmt: "break"

+
+
+
+

5.3.2. Semantics

+
+

If the break statement is outside of a loop body, an error is raised.

+
+
+

At execution, the enclosing loop body stops executing, and execution is continued after the enclosing loop.

+
+
+
+
+

5.4. Continue statement

+
+

A continue statement immediately skips the rest of the loop body, continuing on to the next iteration or ending the loop.

+
+
+

5.4.1. Syntax

+
+

Rule:

+
+
+

continue_stmt: "continue"

+
+
+
+

5.4.2. Semantics

+
+

If the continue statement is outside of a loop body, an error is raised.

+
+
+

At execution:

+
+
+
    +
  1. +

    The enclosing loop body stops executing.

    +
    2. +
  2. +

    If the loop is a for loop, the loop variable is incremented.

    +
    3. +
  3. +

    The loop condition is re-evaluated.

    +
    4. +
  4. +

    The loop continues as specified based on the result of the loop condition, either running the body or ending the loop.

    +
    5. +
+
+
+
+
+
+
+

6. Assert statement

+
+
+

An assert statement evaluates a Boolean expression and halts the program if the expression evaluates to False.

+
+
+

6.1. Syntax

+
+

Rule:

+
+
+

assert_stmt: "assert" expr ["," expr]

+
+
+

Name:

+
+
+

assert_stmt: "assert" condition "," exit_code

+
+
+

condition and exit_code are resolved in the value name group.

+
+
+
+

6.2. Semantics

+
+

If condition cannot be coerced to bool, an error is raised.

+
+
+

If exit_code is provided, and cannot be coerced to U8, an error is raised.

+
+
+

At execution, if condition evaluates to False:

+
+
+
    +
  1. +

    If exit_code is provided, evaluate it and display its value to the user.

    +
    2. +
  2. +

    If exit_code is not provided, display a generic error code to the user.

    +
    3. +
  3. +

    Halt the program.

    +
    4. +
+
+
+
+
+
+

7. Check statement

+
+
+

The check statement executes a block of code if a Boolean expression evaluates to True for a duration of time, checking with a configurable frequency and timing out at a configurable time.

+
+
+

7.1. Syntax

+
+

Rule:

+
+
+

check_stmt: "check" expr check_clause* ":" stmt_list ["timeout" ":" stmt_list]

+
+
+

check_clause: "timeout" expr | "persist" expr | "period" expr

+
+
+

The clauses can appear in any order, and can be spread across multiple indented lines (with the colon after the last clause).

+
+
+

Name:

+
+
+

check_stmt: "check" condition "timeout" timeout "persist" persist "period" period ":" body "timeout" ":" timeout_body

+
+
+

condition, timeout, persist, and period are resolved in the value name group.

+
+
+
+

7.2. Semantics

+
+

If condition cannot be coerced to bool, an error is raised.

+
+
+

If timeout is provided, and cannot be coerced to Fw.TimeIntervalValue, an error is raised.

+
+
+

If persist or period is provided, and they cannot be coerced to Fw.TimeIntervalValue, an error is raised.

+
+
+

At execution:

+
+
+
    +
  1. +

    If provided, timeout, persist and period are evaluated and stored. `timeout’s stored value is the current time plus the evaluated interval.

    +
    2. +
  2. +

    If persist is not provided, its stored value is a zero-duration Fw.TimeIntervalValue.

    +
    3. +
  3. +

    If period is not provided, its stored value is a one-second Fw.TimeIntervalValue.

    +
    4. +
  4. +

    If timeout was provided and the current time is greater than `timeout’s stored value, the check times out.

    +
    5. +
  5. +

    Evaluate condition.

    +
    6. +
  6. +

    If condition has evaluated to True for duration greater than or equal to persist’s stored value, execute `body, then continue execution after the check statement.

    +
    7. +
  7. +

    Otherwise, sleep for `period’s stored duration.

    +
    8. +
  8. +

    Go to step 4.

    +
    9. +
+
+
+

If the check times out during execution:

+
+
+
    +
  1. +

    If timeout_body is provided, execute it.

    +
    2. +
  2. +

    Execution continues after the check statement.

    +
    3. +
+
+
+
+
+

Not providing persist, or providing a zero-duration persist, means the condition only needs to evaluate to True once. +The timeout defaults to never, and the frequency defaults to once per second.

+
+
+
+
+

If at any point during execution, two times which are incomparable are attempted to be compared, the check statement will halt the program as if by an assertion, and display an error code.

+
+
+
+
+
+

8. Imports

+
+
+
+
+

Note: The import statement is not yet implemented.

+
+
+
+
+

An import statement compiles another Fpy source file, called a module, and makes the module’s definitions available in the importing file under a namespace.

+
+
+

8.1. Syntax

+
+

Rule:

+
+
+

import_stmt: "import" name ("." name)*

+
+
+

Name:

+
+
+

import_stmt: "import" module_path

+
+
+

The module path is the entire dotted sequence of names. The first name of a module path is its root segment, and the last is its leaf segment.

+
+
+

An import statement is only valid outside an indentation block.

+
+
+

Tests: 1, 2

+
+
+
+

8.2. Module resolution

+
+

The import search path is an ordered list of directories provided by the environment in which the compiler is invoked.

+
+
+
+
+

In the command-line compiler, the import search path is the directory containing the file being compiled, followed by each directory passed with -i/--include, in order.

+
+
+
+
+

A module path s_0.s_1. …​ .s_n resolves in a directory dir if the file dir/s_0/s_1/…​/s_n.fpy exists.

+
+
+

Tests: 1, 2, 3, 4

+
+
+

The module path of an import statement is resolved in each directory of the import search path, in order. The file from the first directory in which it resolves is the imported module.

+
+
+

Tests: 1, 2, 3, 4

+
+
+

If the module path resolves in no directory of the import search path, an error is raised.

+
+
+

Tests: 1, 2, 3, 4, 5, 6

+
+
+
+
+

Every segment except the leaf names a plain directory; no init.fpy-style marker file is required. Because only the leaf’s .fpy file satisfies an import, a file foo.fpy always takes precedence over a sibling directory foo/ for import foo, while import foo.bar descends into the directory foo/ regardless of whether foo.fpy exists. Importing a name that resolves only to a directory is an error: a directory has no code to import.

+
+
+
+
+
+

8.3. Semantics

+
+

If the imported module fails to parse or compile, an error is raised.

+
+
+

Tests: 1

+
+
+
+
+

The diagnostic should point into the imported file, not at the import statement.

+
+
+
+
+

If the imported module declares one or more sequence arguments, an error is raised.

+
+
+

Tests: 1

+
+
+
+
+

A sequence() directive with no arguments does not prevent a file from being imported.

+
+
+
+
+

Tests: 1

+
+
+

An imported module may itself contain import statements. The semantics of this section apply to them recursively, with the module in the role of the importing file. All import statements in a compilation resolve against the same import search path.

+
+
+

Tests: 1

+
+
+

If a module transitively imports itself, an error is raised.

+
+
+

Tests: 1, 2, 3

+
+
+

The import statement introduces the module path as a chain of namespaces in the global scope. Each definition at the top level of the module adds its symbol to the leaf namespace: a symbol x defined in module a.b.c is named a.b.c.x in the importing file, and is not accessible under any shorter name.

+
+
+

Tests: 1, 2, 3, 4, 5

+
+
+

Per the name group rules, these namespaces exist only in the name groups of the symbols they contain.

+
+
+

Tests: 1

+
+
+

If a name introduced by an import statement is already mapped, in the same name group and the same scope or namespace, to anything other than a namespace introduced by another import statement, an error is raised.

+
+
+

Tests: 1, 2

+
+
+
+
+

Namespaces introduced by imports merge: after import pkg.a and import pkg.b, the namespace pkg contains both a and b. Because name groups do not intersect, an imported module conflicts only with names in the groups it occupies: a variable lib may coexist with an imported module lib that defines only functions.

+
+
+
+
+

Tests: 1

+
+
+

Names within the module are resolved in the module’s own global scope. Symbols of the importing file are not visible in the module, and modules imported by the module are not visible in the importing file.

+
+
+

Tests: 1, 2

+
+
+

If the same module path is imported more than once in the same file, an error is raised.

+
+
+

Tests: 1

+
+
+
+
+

This rule is per-file: a file and a module it imports may each import the same module.

+
+
+
+
+

Tests: 1

+
+
+

If the imported module contains top-level statements other than function definitions and import statements, the import-side-effects warning is emitted.

+
+
+

Tests: 1, 2, 3, 4, 5

+
+
+

At execution, the imported module’s top-level statements execute as part of the importing sequence, at the position of the import statement, in order.

+
+
+

Tests: 1, 2

+
+
+
+
+

Importing is compile-time source inlining: the imported module does not exist in the compiled output, and no files are resolved or loaded at run time. This is what motivates the import-side-effects warning: a file written to run as a standalone sequence typically has top-level commands, and importing it splices that code into the importing sequence, which is rarely intended. A file meant to be imported should contain only definitions. Note that a top-level variable definition is such a side effect (its initialization executes at the import site), while still defining a namespaced symbol: counter: U32 = 5 in module m warns, and is thereafter accessible as m.counter.

+
+
+
+
+
+
+
+

9. Callables

+
+
+

A callable is a symbol with parameters and a return type which can be evaluated by being called.

+
+
+
+
+

Evaluation of a callable always refers to evaluation of a function call expression, where the function is that callable. +Because callable evaluation is always via a function call expression, when talking about the evaluation of a callable, it is assumed that the evaluation semantics of the function call expression have already occurred. Specifically, the arguments have already been evaluated from left to right.

+
+
+
+
+

Callables can be divided into four categories:

+
+
+ +
+
+

9.1. Commands

+
+

A command is a callable with an associated F-Prime command.

+
+
+

All F-Prime commands in the dictionary have a corresponding Fpy command.

+
+
+

All Fpy commands are globally-scoped.

+
+
+

The fully-qualified name of an Fpy command is the same as the corresponding F-Prime command.

+
+
+

The parameter names and types of an Fpy command are the same as the corresponding F-Prime command.

+
+
+
+
+

The F-Prime specification requires that all command parameter types be serializable.

+
+
+
+
+

The return type of all commands is Fw.CmdResponse.

+
+
+
+
+

Throughout the specification, a "command" means an Fpy command.

+
+
+
+
+

9.1.1. Command evaluation

+
+

Command evaluation is performed as follows:

+
+
+
    +
  1. +

    The argument values are serialized and arranged into the F-Prime command binary format.

    +
    2. +
  2. +

    The binary command is dispatched to the F-Prime system.

    +
    3. +
  3. +

    Execution blocks until the command response comes back from the F-Prime system.

    +
    4. +
  4. +

    The expression evaluates to a value of Fw.CmdResponse corresponding to that command response.

    +
    5. +
+
+
+
+

9.1.2. Command responses

+
+

A command response is a value of type Fw.CmdResponse.

+
+
+

Fw.CmdResponse must be defined in the dictionary, otherwise, an error is raised. +TODO gracefully handle when it’s missing.

+
+
+
+
+

9.2. Builtin functions

+
+

A builtin function is a callable whose behavior is explicit in the specification.

+
+
+

9.2.1. exit

+
+
Signature:
+
+

exit(exit_code: I32)

+
+
+
+
Semantics
+
+

At evaluation:

+
+
+
    +
  1. +

    If exit_code evaluates to a non-zero value, the sequence ends with an error.

    +
    2. +
  2. +

    Otherwise, the sequence ends without error.

    +
    3. +
+
+
+
+
+

9.2.2. sleep

+
+
Signature
+
+

sleep(seconds: U32 = 0, useconds: U32 = 0)

+
+
+
+
Semantics
+
+

At evaluation, the program sleeps for a duration of seconds seconds and useconds microseconds.

+
+
+
+
+

9.2.3. sleep_until

+
+
Signature
+
+

sleep_until(wakeup_time: Fw.Time)

+
+
+
+
Semantics
+
+

At evaluation, the program sleeps until the given wakeup_time.

+
+
+
+
+

9.2.4. now

+
+
Signature
+
+

now() → Fw.Time +===== Semantics

+
+
+

At evaluation, the function call evaluates to a Fw.Time value representing the current time. +TODO this really should be linked to the FpySequencer spec to say exactly where it gets this, etc. we also need engineering details

+
+
+
+
+
+

9.3. Builtin libraries

+ +
+
+

9.4. Time functions

+
+

TODO Fpy provides builtin functions for comparing and manipulating time values:

+
+
+
    +
  • +

    time_cmp(lhs: Fw.Time, rhs: Fw.Time) → I8: compares two absolute times. Returns -1 if lhs occurs before rhs, 0 if they are the same moment, 1 if lhs occurs after rhs, or 2 if the time bases differ (incomparable).

    +
    • +
  • +

    time_interval_cmp(lhs: Fw.TimeIntervalValue, rhs: Fw.TimeIntervalValue) → I8: compares two time intervals. Returns -1 if lhs is a shorter duration than rhs, 0 if they are the same duration, or 1 if lhs is a longer duration than rhs.

    +
    • +
  • +

    time_sub(lhs: Fw.Time, rhs: Fw.Time) → Fw.TimeIntervalValue: subtracts two absolute times, producing a time interval. Asserts that both times have the same time base and that lhs occurs after rhs (no negative intervals).

    +
    • +
  • +

    time_add(lhs: Fw.Time, rhs: Fw.TimeIntervalValue) → Fw.Time: adds a time interval to an absolute time, producing a new absolute time. Asserts that the result does not overflow.

    +
    • +
+
+
+

These functions are implemented in Fpy itself (see src/fpy/builtin/time.fpy) and are automatically available in all sequences.

+
+
+
+

9.5. Constructors

+
+

TODO Structs, arrays, and Fw.Time expose constructors whose callable name is the fully qualified type name. Their arguments correspond to the members in definition order (struct fields by name, array elements as e0, e1, …​, and Fw.Time with time_base, time_context, seconds, useconds). A constructor call serializes the provided values into a new instance of that type.

+
+
+
+

9.6. Numeric casts

+
+

TODO Each concrete numeric type provides a callable whose name matches the type (for example U16(value) or F64(value)). Casts accept exactly one numeric argument. Unlike implicit coercion, casts always force the operand into the target type even when this requires narrowing; range checks are suppressed and the value is truncated or rounded if necessary. See Casting for details.

+
+
+
+
+
+

10. Types

+
+
+

A type is a set of values.

+
+
+

The values of a type are unique to that type.

+
+
+
+
+

In other words, there are no union types and there is no type inheritance.

+
+
+
+
+

New types cannot be defined by the program.

+
+
+

A serializable type is a type whose values can be expressed in a binary format.

+
+
+

A constant-sized type is a serializable type whose binary form always has the same length in bytes.

+
+
+

A numeric type is a primitive numeric type, or the internal Int or Float types.

+
+
+
+
+

Right now, the only serializable but non-constant-sized type are the dictionary string types.

+
+
+
+
+

Types can be divided into three categories:

+
+
+
    +
  • +

    Primitive types

    +
    • +
  • +

    Internal types

    +
    • +
  • +

    Dictionary types

    +
    • +
+
+
+

10.1. Primitive types

+
+

Primitive types are types which are always present in the global scope.

+
+
+
+
+

That is, they do not have to be in the F-Prime dictionary to be referenced by name in the program. +Because they are present in the global scope, we will use their associated name in the global scope to refer to them throughout this specification. For instance, when we say type U16, we are talking about the type in the global scope with name U16.

+
+
+
+
+

All primitive types are serializable, constant-sized types.

+
+
+

The list of primitive types is:

+
+
+ +
+
+

10.1.1. Primitive numeric types

+
+

U8, U16, U32, and U64 are the primitive unsigned integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard binary representation of unsigned integers.

+
+
+

I8, I16, I32, and I64 are the primitive signed integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard two’s complement representation of signed integers.

+
+
+

F32, and F64 are the primitive IEEE floating-point types with bitwidths 32 and 64, respectively.

+
+
+
+
+

There are other numerical types such as Int or Float which are not primitive.

+
+
+
+
+
+

10.1.2. Boolean type

+
+

bool is a primitive type whose only values may be the Boolean literals True and False.

+
+
+

TODO make sure that Fw.Time is counted as a dictionary type, but one which is required to be in the dict?

+
+
+
+
+

10.2. Type aliases

+
+

Loop var type is an alias for I64. +Array index type is an alias for I64.

+
+
+
+

10.3. Internal types

+
+

Internal types are types which are never present in the global scope.

+
+
+
+
+

That is, they cannot be referenced by name in the program.

+
+
+
+
+

No internal types are serializable types.

+
+
+

Int is an internal type whose values are integers of arbitrary precision.

+
+
+

Float is an internal type whose values are decimals per the Python decimal implementation.

+
+
+

The precision of Float is 30 decimal places.

+
+
+

String is an internal type whose values are strings of arbitrary length.

+
+
+

Range is an internal type whose values are pairs of an lower and upper bound of loop var type.

+
+
+

Nothing is an internal type which has no values.

+
+
+
+

10.4. Dictionary types

+
+

Dictionary types are types defined in the F-Prime dictionary.

+
+
+
+
+

Because the semantics of these types is defined in the FPP specification, there is some overlap here. This specification just addresses the semantics of these types as relevant to Fpy.

+
+
+
+
+

All dictionary types are serializable types.

+
+
+

Dictionary types can be divided into three categories:

+
+
+ +
+
+

10.4.1. Structs

+
+

A struct is a category of dictionary type defined by an ordered list of members.

+
+
+

A member is a pair of a name and a serializable type.

+
+
+

A struct may not have two members with the same name. +TODO is this rule necessary? This is enforced upstream by FPP

+
+
+

The binary form of a struct value is the concatenated binary forms of its member values, in order.

+
+
+
+
+

If any of a struct’s members are non-constant-sized types, the struct is a non-constant-sized type.

+
+
+
+
+
+

10.4.2. Arrays

+
+

An array is a category of dictionary type defined by a non-negative integer length, and an element type.

+
+
+

The element type of an array type is the type of its elements.

+
+
+

An element is a value of element type at an index in an array.

+
+
+

The binary form of an array value is the concatenated binary form of its elements, in order.

+
+
+
+
+

If the element type is a non-constant-sized type, the array is a non-constant-sized type.

+
+
+
+
+
+

10.4.3. Enums

+
+

An enum is a category of dictionary type whose values are a finite set of enum constants.

+
+
+

An enum constant is a pair of a name and a value of the enum’s representation type.

+
+
+

An enum representation type is the primitive integer type associated with the enum constants' values.

+
+
+

The binary form of an enum constant is the binary form of its integer value.

+
+
+
+

10.4.4. Dictionary strings

+
+

A dictionary string is a category of dictionary type whose values are strings.

+
+
+

Dictionary strings are non-constant-sized types.

+
+
+
+
+

10.5. Fields

+
+

A field-based type is a type defined by its fields.

+
+
+

A field of a type is a name-and-type pair

+
+
+

An array is a category of type with

+
+
+

A struct is a category of type

+
+
+

is an array element or a struct member.

+
+
+

The field base of a field is the first non-field parent of a field.

+
+
+
+
+

For instance, the field base of a.b.c, if a were a variable and b and c were fields, would be a.

+
+
+
+
+
+

10.6. Populating dictionary types

+
+

For each type T with fully-qualified name F.Q.N encountered in the F-Prime dictionary, that type will be present in the global scope with the same fully-qualified name.

+
+
+
+
+
+

11. Expressions

+
+
+

An expression can be evaluated to produce a value of a type.

+
+
+

A constant expression is an expression which can be evaluated without running the program.

+
+
+

11.1. Literals

+
+

A literal is an expression whose value is explicit in the source code.

+
+
+

All literal expressions are constant expressions.

+
+
+

11.1.1. Integer literals

+
+
Decimal literal syntax
+
+

Rule:

+
+
+
+
DEC_LITERAL:   "1".."9" ("_"?  "0".."9")*
+           |   "0"      ("_"?  "0")* /(?![1-9xX])/
+
+
+
+
+
Hexadecimal literal syntax
+
+

Rule:

+
+
+

HEX_LITERAL: ("0x" | "0X") ("_"? /[0-9a-fA-F]/)+

+
+
+
+
Semantics
+
+

Integer literals have type Int.

+
+
+
+
+

11.1.2. Float literals

+
+
Syntax
+
+
+
_SPECIAL_DEC: "0".."9" ("_"?  "0".."9")*
+
+DECIMAL: "." _SPECIAL_DEC | _SPECIAL_DEC "." _SPECIAL_DEC
+_EXP: ("e"|"E") ["+" | "-"] _SPECIAL_DEC
+FLOAT_LITERAL: _SPECIAL_DEC _EXP | DECIMAL _EXP?
+
+
+
+

Float literals have type Float.

+
+
+

A float literal is rounded to the nearest value of type Float.

+
+
+
+
+

11.1.3. String literals

+
+
Syntax
+
+

Rule:

+
+
+

STRING_LITERAL: /("(?!"").?(?<!\\)(\\\\)?"|'(?!'').?(?<!\\)(\\\\)?')/i

+
+
+
+
Semantics
+
+

String literals have type String.

+
+
+
+
+

11.1.4. Boolean literals

+
+
Syntax
+
+

BOOLEAN_LITERAL: "True" | "False" +===== Semantics

+
+
+

Boolean literals have type bool

+
+
+
+
+
+

11.2. Member access expression

+
+

11.2.1. Syntax

+
+

Rule:

+
+
+

member_access_expr: expr "." name

+
+
+

Name:

+
+
+

member_access_expr: parent "." member

+
+
+
+

11.2.2. Semantics

+
+

If parent is not an expression, an error is raised.

+
+
+
+
+

Namespaces, types names, and function names are valid expressions syntactically, but not semantically. Thus, trying to access a member of either of these symbols will raise an error.

+
+
+
+
+

If the type of parent is not a struct, an error is raised.

+
+
+

If the type of parent is not constant-sized, an error is raised.

+
+
+

If member is not a member of the type of parent, an error is raised.

+
+
+

The type of a member access is the type of the member in the type of parent.

+
+
+

At evaluation:

+
+
+
    +
  1. +

    The parent is evaluated.

    +
    2. +
  2. +

    The member access expression evaluates to the value of the member in the parent value.

    +
    3. +
+
+
+
+
+

11.3. Function call expression

+
+

11.3.1. Syntax

+
+

Rule:

+
+
+
+
func_call: expr "(" [arguments] ")"`
+arguments: argument ("," argument)*
+argument: NAME "=" expr -> named_argument
+        | expr -> positional_argument
+
+
+
+

Name:

+
+
+

func_call: func "(" arguments ")"

+
+
+

func is resolved in the callable name group.

+
+
+

All argument expressions are resolved in the value name group.

+
+
+
+

11.3.2. Semantics

+
+

At evaluation:

+
+
+
    +
  1. +

    Each argument is evaluated from left to right.

    +
    2. +
  2. +

    The behavior defined in the semantics for the callable referenced by func is

    +
    3. +
+
+
+
Range semantics
+
+

The range operator is ...

+
+
+

If lhs or rhs cannot be coerced to loop var type, an error is raised.

+
+
+
+
+
+
+
+

12. Execution

+
+
+

12.1. Control flow

+
+

A branch is a block of code which conditionally executes.

+
+
+
+
+

That is, whether or not that code executes depends on some expression.

+
+
+
+
+

The list of statements which have branches is:

+
+
+ +
+
+
+

12.2. Sleeping

+
+

The program may sleep until an absolute time called the wakeup time.

+
+
+

While the current time is before the wakeup time, no statements may execute

+
+
+

The program may also sleep for a time duration. This is the same as sleeping with a wakeup time of now, plus the specified duration.

+
+
+
+
+
+ + + + + \ No newline at end of file diff --git a/docs/spec/01-names-and-scopes.adoc b/docs/spec/01-names-and-scopes.adoc new file mode 100644 index 0000000..901afc9 --- /dev/null +++ b/docs/spec/01-names-and-scopes.adoc @@ -0,0 +1,140 @@ +== Names and scopes + +=== Names + +`name: /\$?[^\W\d]\w*/` + +A *name* is a string consisting of letters, underscores or digits. The first character may not be a digit. + +The first character may optionally be `$`, in which case the name is considered *escaped*. An escaped name is the same as an unescaped name, except in that it always lexes as a name, even if it is a <>. + +=== Reserved words + +A *reserved word* is a word which cannot be used as an <>. + +The list of reserved words is: + +* `assert` +* `break` +* `check` +* `continue` +* `def` +* `for` +* `if` +* `import` +* `not` +* `pass` +* `return` +* `while` + +=== Symbols + +A *symbol* is a language construct that can be referred to by a name in the program. + +The following language constructs may be symbols: + +* <> +* <> +* <> +* <> +* telemetry channels +* parameters +* enum constants +TODO members? +TODO you're selecting a member of a value--not referring to a static member. when you say a.b you're asking for a comp to be performed + +=== Scopes + +A *scope* is a mapping of names to symbols, accessed via some region of the source code. +TODO a scope IS a REGION + +The *global scope* is the scope accessible throughout the entire source code. + +Each <> has a *function scope*, accessible in its body. + +The *resolving scope* is the most specific scope that some part of the source code has access to. + +Scopes may have a *parent scope*: + +* The parent scope of a function scope is the global scope. +* The global scope does not have a parent scope. + +=== Name groups + +Scopes are divided into *name groups*. + +The list of name groups is: + +* The *<> name group* +* The *<> name group* +* The *<> name group* + +Each name group only contains names which map to their particular language construct, or namespaces with the same property, recursively. +TODO Maybe could remove this line + +TODO explain why we need this +Name groups do not intersect. + +> This means that the names of callables, types and values never conflict. WITH EACH OTHER + +Name groups are accessed via syntactic context. +TODO this is really just an explanation + +TODO type names should NOT be expressions + + +TODO A.b is an expr, A is an expr, b is not an expression. A.b is a dot expression. b is not an expression, it's part of a +TODO Does it refer to something that has a scope, or does it refer to something that has members. + +> For instance, the type name group is accessible anywhere in the source code where a type name is expected, such as a <> type annotation, or a <> return type. + +The *resolving name group* is the name group that a name should be resolved in, based on its syntactic context. + +=== Namespaces + +A *namespace* is a mapping of names to symbols, associated with a name. + +=== Qualified names + +A *qualified name* is one of: + +* A name +* A qualified name, followed by a `.`, followed by a name + +The *qualifier* is the qualified name to the left of the `.`. + +To resolve a qualified name in a name group: + +. If there is no qualifier: +.. Resolve the name in the resolving scope. +.. If the name fails to be resolved, resolve the name in the parent scope of the resolving scope. +. Otherwise: +.. Resolve the qualifier. +.. If the qualifier is an expression, resolution is handled by the rules of <>. +.. If the qualifier is not a namespace, an error is raised. +.. Resolve the name in the qualifier namespace. + +If at any point a name fails to be resolved, an error is raised, unless otherwise specified. + +A *fully-qualified name* is a qualified name which is not itself a qualifier. +TODO names are semantic, ident is syntactic +TODO you can't actually tell at syntax level what is a fqn +If a fully-qualified name resolves to a namespace, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L343[1,title="test/fpy/test_imports.py::TestImportNamespaceIsolation::test_module_name_not_usable_as_value"] + +TODO you can think of the dict as "importing definitions" + + +> Namespace symbols cannot be used anywhere, so this forces names to resolve to something "useful" + +TODO I'm not sure this is clear what this means. The idea here is that the full qualified name should always reference SOMETHING--cannot just put Svc in place of a type, even though Svc does resolve in type name group and global scope. + +=== Definitions + +A *definition* is a language construct that introduces a name-to-<> mapping as apart of a <> and <>. + +The list of definitions is: + +* <> +* <> diff --git a/docs/spec/02-variables.adoc b/docs/spec/02-variables.adoc new file mode 100644 index 0000000..71fa972 --- /dev/null +++ b/docs/spec/02-variables.adoc @@ -0,0 +1,152 @@ +== Variables + +A *variable* is a symbol with a static <> and a dynamic, mutable <>. +todo maybe call it dynamic? + +TODO: give a list of statements + +=== Variable definition + +A *variable definition statement* introduces a name-to-variable mapping in its resolving scope. + +==== Syntax + +Rule: + +`variable_declare_stmt: name ":" qualified_name "=" expr` + +Name: + +`variable_declare_stmt: lhs ":" type_ann "=" rhs` + +`lhs` and `rhs` are resolved in the value name group. + +`type_ann` is resolved in the type name group. + +==== Semantics + +If `lhs` resolves to a previously-defined symbol, an error is raised. + +> This prevents redefining a variable. + +If `rhs` cannot be coerced to type `type_ann`, an error is raised. + +The new variable has type `type_ann`. It is added to the resolving scope under name `lhs`. + +At execution, `rhs` is <> and <> to type `type_ann`. This becomes the variable's initial value. + +For statements following this, the variable `lhs` is considered defined. + +=== Variable assignment + +A *variable assignment statement* mutates the value of a variable. + +==== Syntax +Rule: + +`variable_assign_stmt: name "=" expr` + +Name: + +`variable_assign_stmt: lhs "=" rhs` + +`lhs` and `rhs` are resolved in the value name group. + +==== Semantics + +If `lhs` does not resolve to a variable, an error is raised. + +> Hereafter, `lhs` refers to the variable named `lhs`. + +If `lhs` has not been defined yet, an error is raised. + +If `rhs` cannot be coerced to `lhs`'s type, an error is raised. + +At execution, `rhs` is <> and <> to `lhs`'s type. This becomes the `lhs`'s new value. + +=== Member assignment + +A *member assignment statement* mutates the value of a <> in a variable. + +==== Syntax +Rule: + +`variable_assign_member_stmt: expr "." name "=" expr` + +Name: + +`variable_assign_member_stmt: parent "." member "=" rhs` + +`parent`, and `rhs` are resolved in the value name group. + +==== Semantics + +If `parent` is not a variable or a <> with a <> that is a variable, an error is raised. + +> This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you're modifying is a variable. + +If `parent`'s type is: + +* not <>, or +* not a <> type, or +* `member` is not a member of `parent`'s type + +... an error is raised. + +If the variable has not been defined yet, an error is raised. + +If `rhs` cannot be coerced to the member's type, an error is raised. + +At execution, `rhs` is <> and <> to the member's type. This becomes the member's new value. + +The value of the variable is unchanged except for the member. + +=== Element assignment + +An *element assignment statement* mutates the value of an link:todo[element] in a variable. + +==== Syntax +Rule: + +`variable_assign_element_stmt: expr "[" expr "]" "=" expr` + +Name: + +`variable_assign_element_stmt: parent "[" item "]" "=" rhs` + +`parent`, `item` and `rhs` are resolved in the value name group. + +==== Semantics + +If `parent` is not a variable or a <> with a link:todo[field base] that is a variable, an error is raised. + +> This allows for setting a field of a field to arbitrary depth, as long as the underlying thing you're modifying is a variable. + +If the variable has not been defined yet, an error is raised. + +If `item` cannot be <> to <>, an error is raised. + +If `parent`'s type is: + +* not <>, or +* not an <> type, or +* `item` is a link:todo[constant] with a value less than 0 or greater than the `parent`'s type length, + +... an error is raised. + +If `rhs` cannot be coerced to the element's type, an error is raised. + +At execution: + +. `rhs` is evaluated and coerced to the element's type +. `item` is <> and coerced to <> +. If `item` is less than zero or greater than the `parent`'s type length, a runtime error is raised +. The element in the `parent` array at the index `item` is set to the result of step 1 + +The value of the variable is unchanged except for the element. + +=== Variable evaluation + +The value produced by <> a variable is the value most recently assigned to that variable, or the initial value if it has only been defined. + +If a variable is evaluated before it has been defined, an error is raised. diff --git a/docs/spec/03-functions.adoc b/docs/spec/03-functions.adoc new file mode 100644 index 0000000..339c2e2 --- /dev/null +++ b/docs/spec/03-functions.adoc @@ -0,0 +1,106 @@ +== Functions + +A *function* is a <> <> with an inner scope, parameters, code and a return <>. + +The *call site* is the location in the source code at which a function is called. + +=== Function parameters + +A *function parameter* is a <> implicitly defined by a function in that function's scope. + +When a function is link:todo[called], each parameter is set to an initial value. + +The initial value may either be from a passed link:todo[argument], or a default value, if one is specified in the function definition. + +> In all other respects, parameters are like normal variables, meaning you can modify them in the function body. + +=== Return types + +The *return type* of a function is the <> of the value returned by that function. If the return type is <>, the function does not return a value. + +=== Returns + +A *return statement* ends the currently executing function, resumes execution at the call site, and optionally returns a value. + +==== Syntax +Rule: + +`return_stmt: "return" [expr]` + +Name: + +`return_stmt: "return" value` + +`value` is resolved in the value name group. + +==== Semantics + +If the return statement is outside of a function body, an error is raised. + +The *enclosing function* of a return statement is the function whose body that return is in. + +If `value` is not provided and the enclosing function's return type is not Nothing, an error is raised. + +If `value` is provided and cannot be <> to the return type of the enclosing function, an error is raised. + +At execution: + +. If provided, `value` is link:todo[evaluated] and <> to the return type of the enclosing function. +. The execution of the function body is stopped, and execution after the function call site resumes. + +=== Function definition + +A *function definition statement* introduces a name-to-<> mapping in the global scope. + +==== Syntax + +Rule: + +[listing] +---- +function_def_stmt: "def" name "(" [parameters] ")" ["->" qualified_name] ":" block +parameters: parameter ("," parameter)* +parameter: name ":" qualified_name ["=" expr] +---- + +Name: + +[listing] +---- +function_def_stmt: "def" name "(" parameters ")" "->" return_type ":" body +parameters: parameter_0 "," parameter_1 ... "," parameter_n +parameter: parameter_name ":" parameter_type "=" parameter_default_value +---- + +A function definition statement is only valid outside an indentation block. + +The parameter `name`s are resolved in the value name group. + +`name` is resolved in the callable name group. + +`return_type` and each of the parameter types are resolved in the type name group. + +==== Semantics + +If `name` resolves to a previously-defined callable, an error is raised. + +A new function <> is created, accessible to the `body` and the parameter `name`s. + +Each parameter is a variable in this new scope. + +> This implies that no two parameters may have the same name, otherwise they would be conflicting variables. + +If the default value of a parameter is not a link:todo[constant], an error is raised. + +If the default value of a parameter cannot be <> to the type of the parameter, an error is raised. + +If a parameter without a default value follows a parameter with a default value, an error is raised. + +If `return_type` is provided, and any link:todo[branch] of the function does not return a value, an error is raised. +TODO need a section on control flow? + +The new function with name `name` is added to the global scope. If `return_type` is not provided, the <> is <>, otherwise the return type is type `return_type`. + +> Because functions can only be defined in the global scope, you cannot declare a function in a function. + +> Functions can be used before they are defined. diff --git a/docs/spec/04-ifs.adoc b/docs/spec/04-ifs.adoc new file mode 100644 index 0000000..38ec9e0 --- /dev/null +++ b/docs/spec/04-ifs.adoc @@ -0,0 +1,29 @@ +== Ifs +An *if statement* conditionally executes blocks of code. + +=== Syntax +Rule: + +`if_stmt: "if" expr ":" stmt_list elifs ["else" ":" stmt_list]` + +`elifs: elif_*` + +`elif_: "elif" expr ":" stmt_list` + +Name: + +`if_stmt: "if" if_condition ":" body elifs "else" ":" else_body` + +`elif_: "elif" elif_condition ":" elif_body` + +`if_condition` and all `elif_condition`s are resolved in the value name group. + +=== Semantics + +If `if_condition` or any `elif_condition` cannot be <> to <>, an error is raised. + +At execution, the conditions will be evaluated one at a time until one evaluates to `True`, starting from `if_condition` and going in order through the `elif_conditions`. + +The body of the first condition to evaluate to `True` is executed, and then execution continues after the if statement. + +If no condition evaluates to `True`, and an `else_body` was provided, that body is executed, and then execution continues after the if statement. diff --git a/docs/spec/05-loops.adoc b/docs/spec/05-loops.adoc new file mode 100644 index 0000000..1afedf6 --- /dev/null +++ b/docs/spec/05-loops.adoc @@ -0,0 +1,119 @@ +== Loops + +A *loop* executes a block of code zero or more times. + +Each loop has a *loop condition*, which is a Boolean expression which, when `True`, allows the loop body to execute. + +The *enclosing loop* is the loop whose body some source code is in. + +The list of loops is: + +* <> +* <> + +=== While loop statement + +A *while statement* executes a block of code in a loop while a condition holds `True`. + +==== Syntax + +Rule: + +`while_stmt: "while" expr ":" stmt_list` + +Name: + +`while_stmt: "while" condition ":" body` + +`condition` is resolved in the value name group. + +==== Semantics + +If `condition` cannot be <> to <>, an error is raised. + +The loop condition of a while loop is the provided `condition`. + +At execution: + +. The loop condition is evaluated. +. If the loop condition is `True`, execute the body, and return to step 1. +. Otherwise, execution continues after the while loop statement. + +=== For loop statement + +A *for loop statement* executes a block of code until a counter reaches an upper bound. + +==== Syntax +Rule: + +`for_stmt: "for" name "in" expr ":" stmt_list` + +Name: + +`for_stmt: "for" loop_var "in" range ":" body` + +`loop_var` and `range` are resolved in the value name group. + +==== Semantics + +The *loop variable* of a for loop is the variable named by `loop_var`. + +If `loop_var` resolves to a previously-defined variable: + +. If the type of that variable is not <>, an error is raised. +. Otherwise, that variable becomes the loop variable of this for loop. + +> This allows reusing the same loop variable name across multiple for loops. + +If `loop_var` does not resolve to a previously-defined variable, a new variable with name `loop_var` and <> is added to the <>, and it becomes the loop variable of this for loop. + +> Nothing prevents you from modifying the loop variable in the loop body. However, this may cause infinite loops, so do this with caution. + +If `range` cannot be <> to <>, an error is raised. + +The loop condition of a for loop is `loop_var < upper_bound`, where `upper_bound` is the upper bound of the `range` expression. + +At execution: + +. `range` is evaluated. +. The loop variable is set to the lower bound of `range`. +. The loop condition is evaluated. +. If the loop condition is `True`, execute the body, increment the value of the `loop_var` by 1, and return to step 1. +. Otherwise, execution continues after the for loop statement. + +> The only possible step size is 1. + +=== Break statement + +A *break statement* stops execution of a loop. + +==== Syntax +Rule: + +`break_stmt: "break"` + +==== Semantics + +If the break statement is outside of a loop body, an error is raised. + +At execution, the enclosing loop body stops executing, and execution is continued after the enclosing loop. + +=== Continue statement + +A *continue statement* immediately skips the rest of the loop body, continuing on to the next iteration or ending the loop. + +==== Syntax +Rule: + +`continue_stmt: "continue"` + +==== Semantics + +If the continue statement is outside of a loop body, an error is raised. + +At execution: + +. The enclosing loop body stops executing. +. If the loop is a <>, the loop variable is incremented. +. The loop condition is re-evaluated. +. The loop continues as specified based on the result of the loop condition, either running the body or ending the loop. diff --git a/docs/spec/06-assert-statement.adoc b/docs/spec/06-assert-statement.adoc new file mode 100644 index 0000000..7364b37 --- /dev/null +++ b/docs/spec/06-assert-statement.adoc @@ -0,0 +1,26 @@ +== Assert statement + +An *assert statement* evaluates a Boolean expression and halts the program if the expression evaluates to `False`. + +=== Syntax +Rule: + +`assert_stmt: "assert" expr ["," expr]` + +Name: + +`assert_stmt: "assert" condition "," exit_code` + +`condition` and `exit_code` are resolved in the value name group. + +=== Semantics + +If `condition` cannot be coerced to <>, an error is raised. + +If `exit_code` is provided, and cannot be coerced to <>, an error is raised. + +At execution, if `condition` evaluates to `False`: + +. If `exit_code` is provided, evaluate it and display its value to the user. +. If `exit_code` is not provided, display a generic error code to the user. +. Halt the program. diff --git a/docs/spec/07-check-statement.adoc b/docs/spec/07-check-statement.adoc new file mode 100644 index 0000000..7d9df3c --- /dev/null +++ b/docs/spec/07-check-statement.adoc @@ -0,0 +1,46 @@ +== Check statement +The *check statement* executes a block of code if a Boolean expression evaluates to `True` for a duration of time, checking with a configurable frequency and timing out at a configurable time. + +=== Syntax +Rule: + +`check_stmt: "check" expr check_clause* ":" stmt_list ["timeout" ":" stmt_list]` + +`check_clause: "timeout" expr | "persist" expr | "period" expr` + +The clauses can appear in any order, and can be spread across multiple indented lines (with the colon after the last clause). + +Name: + +`check_stmt: "check" condition "timeout" timeout "persist" persist "period" period ":" body "timeout" ":" timeout_body` + +`condition`, `timeout`, `persist`, and `period` are resolved in the value name group. + +=== Semantics + +If `condition` cannot be <> to <>, an error is raised. + +If `timeout` is provided, and cannot be coerced to link:todo[`Fw.TimeIntervalValue`], an error is raised. + +If `persist` or `period` is provided, and they cannot be coerced to link:todo[`Fw.TimeIntervalValue`], an error is raised. + +At execution: + +. If provided, `timeout`, `persist` and `period` are evaluated and stored. `timeout`'s stored value is the current time plus the evaluated interval. +. If `persist` is not provided, its stored value is a zero-duration `Fw.TimeIntervalValue`. +. If `period` is not provided, its stored value is a one-second `Fw.TimeIntervalValue`. +. If `timeout` was provided and the current time is link:todo[greater] than `timeout`'s stored value, the check times out. +. Evaluate `condition`. +. If `condition` has evaluated to `True` for duration greater than or equal to `persist`'s stored value, execute `body`, then continue execution after the check statement. +. Otherwise, link:todo[sleep] for `period`'s stored duration. +. Go to step 4. + +If the check times out during execution: + +. If `timeout_body` is provided, execute it. +. Execution continues after the check statement. + +> Not providing `persist`, or providing a zero-duration `persist`, means the `condition` only needs to evaluate to `True` once. +> The timeout defaults to never, and the frequency defaults to once per second. + +If at any point during execution, two times which are link:todo[incomparable] are attempted to be compared, the check statement will halt the program as if by an <>, and display an error code. diff --git a/docs/spec/08-imports.adoc b/docs/spec/08-imports.adoc new file mode 100644 index 0000000..a17ea02 --- /dev/null +++ b/docs/spec/08-imports.adoc @@ -0,0 +1,103 @@ +== Imports + +> *Note:* The import statement is not yet implemented. + +An *import statement* compiles another Fpy source file, called a *module*, and makes the module's <> available in the importing file under a <>. + +=== Syntax + +Rule: + +`import_stmt: "import" name ("." name)*` + +Name: + +`import_stmt: "import" module_path` + +The *module path* is the entire dotted sequence of names. The first name of a module path is its *root segment*, and the last is its *leaf segment*. + +An import statement is only valid outside an indentation block. + +_Tests:_ link:test/fpy/test_imports.py#L535[1,title="test/fpy/test_imports.py::TestImportOnlyAtTopLevel::test_import_inside_if_block_fails"], link:test/fpy/test_imports.py#L552[2,title="test/fpy/test_imports.py::TestImportOnlyAtTopLevel::test_import_inside_function_fails"] + +=== Module resolution + +The *import search path* is an ordered list of directories provided by the environment in which the compiler is invoked. + +> In the command-line compiler, the import search path is the directory containing the file being compiled, followed by each directory passed with `-i`/`--include`, in order. + +A module path `s_0.s_1. ... .s_n` *resolves* in a directory `dir` if the file `dir/s_0/s_1/.../s_n.fpy` exists. + +_Tests:_ link:test/fpy/test_imports.py#L705[1,title="test/fpy/test_imports.py::TestImportDottedPaths::test_single_dotted_import"], link:test/fpy/test_imports.py#L725[2,title="test/fpy/test_imports.py::TestImportDottedPaths::test_deeply_nested_dotted_import"], link:test/fpy/test_imports.py#L801[3,title="test/fpy/test_imports.py::TestImportPackagePrecedence::test_module_file_beats_namespace_directory"], link:test/fpy/test_imports.py#L815[4,title="test/fpy/test_imports.py::TestImportPackagePrecedence::test_package_dir_used_for_dotted_descent"] + +The module path of an import statement is resolved in each directory of the import search path, in order. The file from the first directory in which it resolves is the imported module. + +_Tests:_ link:test/fpy/test_imports.py#L856[1,title="test/fpy/test_imports.py::TestImportSearchDirs::test_module_found_in_later_search_dir"], link:test/fpy/test_imports.py#L871[2,title="test/fpy/test_imports.py::TestImportSearchDirs::test_first_search_dir_shadows_later"], link:test/fpy/test_imports.py#L887[3,title="test/fpy/test_imports.py::TestImportSearchDirs::test_search_order_respects_dir_order"], link:test/fpy/test_imports.py#L903[4,title="test/fpy/test_imports.py::TestImportSearchDirs::test_dotted_module_resolved_across_search_dirs"] + +If the module path resolves in no directory of the import search path, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L246[1,title="test/fpy/test_imports.py::TestImportErrors::test_missing_module_is_an_error"], link:test/fpy/test_imports.py#L919[2,title="test/fpy/test_imports.py::TestImportSearchDirs::test_no_search_dirs_cannot_resolve"], link:test/fpy/test_imports.py#L763[3,title="test/fpy/test_imports.py::TestImportDottedPaths::test_missing_leaf_in_existing_package_is_error"], link:test/fpy/test_imports.py#L828[4,title="test/fpy/test_imports.py::TestImportPackagePrecedence::test_bare_package_import_is_error"], link:test/fpy/test_imports.py#L841[5,title="test/fpy/test_imports.py::TestImportPackagePrecedence::test_dotted_leaf_package_import_is_error"], link:test/fpy/test_imports.py#L297[6,title="test/fpy/test_imports.py::TestImportFileErrors::test_import_path_is_a_directory_fails"] + +> Every segment except the leaf names a plain directory; no `__init__.fpy`-style marker file is required. Because only the leaf's `.fpy` file satisfies an import, a file `foo.fpy` always takes precedence over a sibling directory `foo/` for `import foo`, while `import foo.bar` descends into the directory `foo/` regardless of whether `foo.fpy` exists. Importing a name that resolves only to a directory is an error: a directory has no code to import. + +=== Semantics + +If the imported module fails to parse or compile, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L279[1,title="test/fpy/test_imports.py::TestImportFileErrors::test_parse_error_in_imported_file_fails"] + +> The diagnostic should point into the imported file, not at the import statement. + +If the imported module declares one or more link:todo[sequence arguments], an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L228[1,title="test/fpy/test_imports.py::TestImportErrors::test_cannot_import_sequence_with_arguments"] + +> A `sequence()` directive with no arguments does not prevent a file from being imported. + +_Tests:_ link:test/fpy/test_imports.py#L254[1,title="test/fpy/test_imports.py::TestImportErrors::test_no_arg_sequence_is_importable"] + +An imported module may itself contain import statements. The semantics of this section apply to them recursively, with the module in the role of the importing file. All import statements in a compilation resolve against the same import search path. + +_Tests:_ link:test/fpy/test_imports.py#L574[1,title="test/fpy/test_imports.py::TestImportTransitive::test_transitive_import_works"] + +If a module transitively imports itself, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L635[1,title="test/fpy/test_imports.py::TestImportCycles::test_self_import_is_cycle_error"], link:test/fpy/test_imports.py#L656[2,title="test/fpy/test_imports.py::TestImportCycles::test_mutual_import_is_cycle_error"], link:test/fpy/test_imports.py#L687[3,title="test/fpy/test_imports.py::TestImportCycles::test_three_way_cycle_error"] + +The import statement introduces the module path as a chain of namespaces in the global scope. Each <> at the top level of the module adds its symbol to the leaf namespace: a symbol `x` defined in module `a.b.c` is named `a.b.c.x` in the importing file, and is not accessible under any shorter name. + +_Tests:_ link:test/fpy/test_imports.py#L85[1,title="test/fpy/test_imports.py::TestImportInlining::test_call_imported_function"], link:test/fpy/test_imports.py#L123[2,title="test/fpy/test_imports.py::TestImportInlining::test_local_and_imported_names_coexist"], link:test/fpy/test_imports.py#L324[3,title="test/fpy/test_imports.py::TestImportNamespaceIsolation::test_imported_symbol_requires_module_prefix"], link:test/fpy/test_imports.py#L362[4,title="test/fpy/test_imports.py::TestImportNamespaceIsolation::test_same_function_name_in_two_modules_no_collision"], link:test/fpy/test_imports.py#L743[5,title="test/fpy/test_imports.py::TestImportDottedPaths::test_dotted_symbol_requires_full_path"] + +Per the <> rules, these namespaces exist only in the name groups of the symbols they contain. + +_Tests:_ link:test/fpy/test_imports.py#L461[1,title="test/fpy/test_imports.py::TestImportNameCollisions::test_import_coexists_with_local_variable"] + +If a name introduced by an import statement is already mapped, in the same name group and the same scope or namespace, to anything other than a namespace introduced by another import statement, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L421[1,title="test/fpy/test_imports.py::TestImportNameCollisions::test_import_collides_with_local_function"], link:test/fpy/test_imports.py#L442[2,title="test/fpy/test_imports.py::TestImportNameCollisions::test_import_collides_with_local_variable"] + +> Namespaces introduced by imports merge: after `import pkg.a` and `import pkg.b`, the namespace `pkg` contains both `a` and `b`. Because name groups do not intersect, an imported module conflicts only with names in the groups it occupies: a variable `lib` may coexist with an imported module `lib` that defines only functions. + +_Tests:_ link:test/fpy/test_imports.py#L780[1,title="test/fpy/test_imports.py::TestImportDottedPaths::test_two_modules_in_same_package_no_collision"] + +Names within the module are resolved in the module's own global scope. Symbols of the importing file are not visible in the module, and modules imported by the module are not visible in the importing file. + +_Tests:_ link:test/fpy/test_imports.py#L392[1,title="test/fpy/test_imports.py::TestImportNamespaceIsolation::test_imported_function_cannot_see_importer_globals"], link:test/fpy/test_imports.py#L602[2,title="test/fpy/test_imports.py::TestImportTransitive::test_transitive_dependency_is_private"] + +If the same module path is imported more than once in the same file, an error is raised. + +_Tests:_ link:test/fpy/test_imports.py#L489[1,title="test/fpy/test_imports.py::TestImportDuplicates::test_duplicate_import_is_error"] + +> This rule is per-file: a file and a module it imports may each import the same module. + +_Tests:_ link:test/fpy/test_imports.py#L506[1,title="test/fpy/test_imports.py::TestImportDuplicates::test_duplicate_across_files_is_allowed"] + +If the imported module contains top-level statements other than function definitions and import statements, the `import-side-effects` warning is emitted. + +_Tests:_ link:test/fpy/test_imports.py#L158[1,title="test/fpy/test_imports.py::TestImportSideEffects::test_side_effecting_import_warns"], link:test/fpy/test_imports.py#L172[2,title="test/fpy/test_imports.py::TestImportSideEffects::test_side_effect_warning_can_be_ignored"], link:test/fpy/test_imports.py#L187[3,title="test/fpy/test_imports.py::TestImportSideEffects::test_side_effect_warning_can_be_escalated"], link:test/fpy/test_imports.py#L202[4,title="test/fpy/test_imports.py::TestImportSideEffects::test_functions_only_module_does_not_warn"], link:test/fpy/test_imports.py#L308[5,title="test/fpy/test_imports.py::TestImportFileErrors::test_empty_module_compiles_without_warning"] + +At execution, the imported module's top-level statements execute as part of the importing sequence, at the position of the import statement, in order. + +_Tests:_ link:test/fpy/test_imports.py#L106[1,title="test/fpy/test_imports.py::TestImportInlining::test_imported_function_runs"], link:test/fpy/test_imports.py#L932[2,title="test/fpy/test_imports.py::TestImportVariables::test_top_level_variable_is_side_effect_and_namespaced"] + +> Importing is compile-time source inlining: the imported module does not exist in the compiled output, and no files are resolved or loaded at run time. This is what motivates the `import-side-effects` warning: a file written to run as a standalone sequence typically has top-level commands, and importing it splices that code into the importing sequence, which is rarely intended. A file meant to be imported should contain only definitions. Note that a top-level variable definition is such a side effect (its initialization executes at the import site), while still defining a namespaced symbol: `counter: U32 = 5` in module `m` warns, and is thereafter accessible as `m.counter`. diff --git a/docs/spec/09-callables.adoc b/docs/spec/09-callables.adoc new file mode 100644 index 0000000..79aab0d --- /dev/null +++ b/docs/spec/09-callables.adoc @@ -0,0 +1,106 @@ +== Callables + +A *callable* is a symbol with parameters and a return <> which can be evaluated by being called. + +> Evaluation of a callable always refers to evaluation of a <>, where the function is that callable. +> Because callable evaluation is always via a function call expression, when talking about the evaluation of a callable, it is assumed that the evaluation semantics of the function call expression have already occurred. Specifically, the arguments have already been evaluated from left to right. + +Callables can be divided into four categories: + +* <> +* <> +* <> +* link:todo[Constructors] + +=== Commands +A *command* is a callable with an associated F-Prime command. + +All F-Prime commands in the dictionary have a corresponding Fpy command. + +All Fpy commands are <>. + +The fully-qualified name of an Fpy command is the same as the corresponding F-Prime command. + +The parameter names and types of an Fpy command are the same as the corresponding F-Prime command. + +> The F-Prime specification requires that all command parameter types be <>. + +The return type of all commands is link:todo[`Fw.CmdResponse`]. + +> Throughout the specification, a "command" means an Fpy command. + +==== Command evaluation + +Command evaluation is performed as follows: + +. The argument values are serialized and arranged into the F-Prime command binary format. +. The binary command is dispatched to the F-Prime system. +. Execution blocks until the link:todo[command response] comes back from the F-Prime system. +. The expression evaluates to a value of `Fw.CmdResponse` corresponding to that command response. + +==== Command responses + +A *command response* is a value of type `Fw.CmdResponse`. + +`Fw.CmdResponse` must be defined in the dictionary, otherwise, an error is raised. +TODO gracefully handle when it's missing. + +=== Builtin functions +A *builtin function* is a callable whose behavior is explicit in the specification. + +==== `exit` +===== Signature: + +`exit(exit_code: I32)` + +===== Semantics +At evaluation: + +. If `exit_code` evaluates to a non-zero value, the sequence ends with an error. +. Otherwise, the sequence ends without error. + +==== `sleep` +===== Signature + +`sleep(seconds: U32 = 0, useconds: U32 = 0)` + +===== Semantics + +At evaluation, the program <> for a duration of `seconds` seconds and `useconds` microseconds. + +==== `sleep_until` +===== Signature + +`sleep_until(wakeup_time: Fw.Time)` + +===== Semantics + +At evaluation, the program <> until the given `wakeup_time`. + +==== `now` +===== Signature +`now() -> Fw.Time` +===== Semantics + +At evaluation, the function call evaluates to a `Fw.Time` value representing the current time. +TODO this really should be linked to the FpySequencer spec to say exactly where it gets this, etc. we also need engineering details + + +=== Builtin libraries + + +=== Time functions +TODO Fpy provides builtin functions for comparing and manipulating time values: + +* `time_cmp(lhs: Fw.Time, rhs: Fw.Time) -> I8`: compares two absolute times. Returns `-1` if `lhs` occurs before `rhs`, `0` if they are the same moment, `1` if `lhs` occurs after `rhs`, or `2` if the time bases differ (incomparable). +* `time_interval_cmp(lhs: Fw.TimeIntervalValue, rhs: Fw.TimeIntervalValue) -> I8`: compares two time intervals. Returns `-1` if `lhs` is a shorter duration than `rhs`, `0` if they are the same duration, or `1` if `lhs` is a longer duration than `rhs`. +* `time_sub(lhs: Fw.Time, rhs: Fw.Time) -> Fw.TimeIntervalValue`: subtracts two absolute times, producing a time interval. Asserts that both times have the same time base and that `lhs` occurs after `rhs` (no negative intervals). +* `time_add(lhs: Fw.Time, rhs: Fw.TimeIntervalValue) -> Fw.Time`: adds a time interval to an absolute time, producing a new absolute time. Asserts that the result does not overflow. + +These functions are implemented in Fpy itself (see `src/fpy/builtin/time.fpy`) and are automatically available in all sequences. + +=== Constructors +TODO Structs, arrays, and `Fw.Time` expose constructors whose callable name is the fully qualified type name. Their arguments correspond to the members in definition order (struct fields by name, array elements as `e0`, `e1`, ..., and `Fw.Time` with `time_base`, `time_context`, `seconds`, `useconds`). A constructor call serializes the provided values into a new instance of that type. + +=== Numeric casts +TODO Each concrete numeric type provides a callable whose name matches the type (for example `U16(value)` or `F64(value)`). Casts accept exactly one numeric argument. Unlike implicit coercion, casts always force the operand into the target type even when this requires narrowing; range checks are suppressed and the value is truncated or rounded if necessary. See <> for details. diff --git a/docs/spec/10-types.adoc b/docs/spec/10-types.adoc new file mode 100644 index 0000000..be73790 --- /dev/null +++ b/docs/spec/10-types.adoc @@ -0,0 +1,149 @@ +== Types + +A *type* is a set of *values*. + +The values of a type are unique to that type. + +> In other words, there are no union types and there is no type inheritance. + +New types cannot be defined by the program. + +A *serializable type* is a type whose values can be expressed in a binary format. + +A *constant-sized type* is a serializable type whose binary form always has the same length in bytes. + +A *numeric type* is a <>, or the <> types. + +> Right now, the only serializable but non-constant-sized type are the <> types. + +Types can be divided into three categories: + +* Primitive types +* Internal types +* Dictionary types + +=== Primitive types + +*Primitive types* are types which are always present in the global scope. + +> That is, they do not have to be in the F-Prime dictionary to be referenced by name in the program. +> Because they are present in the global scope, we will use their associated name in the global scope to refer to them throughout this specification. For instance, when we say type `U16`, we are talking about the type in the global scope with name `U16`. + +All primitive types are serializable, constant-sized types. + +The list of primitive types is: + +* All <> +* The <> + +==== Primitive numeric types +`U8`, `U16`, `U32`, and `U64` are the primitive unsigned integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard binary representation of unsigned integers. + +`I8`, `I16`, `I32`, and `I64` are the primitive signed integer types with bitwidths 8, 16, 32 and 64, respectively. They use the standard two's complement representation of signed integers. + +`F32`, and `F64` are the primitive IEEE floating-point types with bitwidths 32 and 64, respectively. + +> There are other numerical types such as <> which are not primitive. + +==== Boolean type +`bool` is a primitive type whose only values may be the link:todo[Boolean literals] `True` and `False`. + +TODO make sure that Fw.Time is counted as a dictionary type, but one which is required to be in the dict? + +=== Type aliases +*Loop var type* is an alias for `I64`. +*Array index type* is an alias for `I64`. + +=== Internal types + +*Internal types* are types which are never present in the global scope. + +> That is, they cannot be referenced by name in the program. + +No internal types are serializable types. + +*Int* is an internal type whose values are integers of arbitrary precision. + +*Float* is an internal type whose values are decimals per the Python link:https://docs.python.org/3/library/decimal.html#module-decimal[decimal] implementation. + +The precision of Float is 30 decimal places. + +*String* is an internal type whose values are strings of arbitrary length. + +*Range* is an internal type whose values are pairs of an lower and upper bound of loop var type. + +*Nothing* is an internal type which has no values. + +=== Dictionary types +*Dictionary types* are types defined in the F-Prime dictionary. + +> Because the semantics of these types is defined in the FPP specification, there is some overlap here. This specification just addresses the semantics of these types as relevant to Fpy. + +All dictionary types are serializable types. + +Dictionary types can be divided into three categories: + +* <> +* <> +* <> +* <> + +==== Structs +A *struct* is a category of dictionary type defined by an ordered list of members. + +A *member* is a pair of a name and a serializable type. + +A struct may not have two members with the same name. +TODO is this rule necessary? This is enforced upstream by FPP + +The binary form of a struct value is the concatenated binary forms of its member values, in order. + +> If any of a struct's members are non-constant-sized types, the struct is a non-constant-sized type. + +==== Arrays + +An *array* is a category of dictionary type defined by a non-negative integer length, and an element type. + +The *element type* of an array type is the type of its elements. + +An *element* is a value of element type at an index in an array. + +The binary form of an array value is the concatenated binary form of its elements, in order. + +> If the element type is a non-constant-sized type, the array is a non-constant-sized type. + +==== Enums + +An *enum* is a category of dictionary type whose values are a finite set of enum constants. + +An *enum constant* is a pair of a name and a value of the enum's representation type. + +An *enum representation type* is the <> associated with the enum constants' values. + +The binary form of an enum constant is the binary form of its integer value. + +==== Dictionary strings + +A *dictionary string* is a category of dictionary type whose values are strings. + +Dictionary strings are non-constant-sized types. + +=== Fields + +A *field-based type* is a type defined by its fields. + +A *field* of a type is a name-and-type pair + +An *array* is a category of type with + +A *struct* is a category of type + +is an <> or a <>. + +The *field base* of a field is the first non-field parent of a field. + +> For instance, the field base of `a.b.c`, if `a` were a variable and `b` and `c` were fields, would be `a`. + +=== Populating dictionary types + +For each type `T` with fully-qualified name `F.Q.N` encountered in the F-Prime dictionary, that type will be present in the global scope with the same fully-qualified name. diff --git a/docs/spec/11-expressions.adoc b/docs/spec/11-expressions.adoc new file mode 100644 index 0000000..0b986fe --- /dev/null +++ b/docs/spec/11-expressions.adoc @@ -0,0 +1,130 @@ +== Expressions + +An *expression* can be evaluated to produce a value of a type. + +A *constant expression* is an expression which can be evaluated without running the program. + +=== Literals + +A *literal* is an expression whose value is explicit in the source code. + +All literal expressions are constant expressions. + +==== Integer literals + +===== Decimal literal syntax + +Rule: + +[listing] +---- +DEC_LITERAL: "1".."9" ("_"? "0".."9")* + | "0" ("_"? "0")* /(?![1-9xX])/ +---- + +===== Hexadecimal literal syntax + +Rule: + +`HEX_LITERAL: ("0x" | "0X") ("_"? /[0-9a-fA-F]/)+` + +===== Semantics + +Integer literals have type <>. + +==== Float literals + +===== Syntax +[listing] +---- +_SPECIAL_DEC: "0".."9" ("_"? "0".."9")* + +DECIMAL: "." _SPECIAL_DEC | _SPECIAL_DEC "." _SPECIAL_DEC +_EXP: ("e"|"E") ["+" | "-"] _SPECIAL_DEC +FLOAT_LITERAL: _SPECIAL_DEC _EXP | DECIMAL _EXP? +---- + +Float literals have type <>. + +A float literal is rounded to the nearest value of type Float. + +==== String literals +===== Syntax + +Rule: + +`STRING_LITERAL: /("(?!"").*?(?>. + +==== Boolean literals +===== Syntax +`BOOLEAN_LITERAL: "True" | "False"` +===== Semantics + +Boolean literals have type <> + +=== Member access expression +==== Syntax + +Rule: + +`member_access_expr: expr "." name` + +Name: + +`member_access_expr: parent "." member` + +==== Semantics + +If `parent` is not an expression, an error is raised. + +> Namespaces, types names, and function names are valid expressions syntactically, but not semantically. Thus, trying to access a member of either of these symbols will raise an error. + +If the type of `parent` is not a <>, an error is raised. + +If the type of `parent` is not <>, an error is raised. + +If `member` is not a member of the type of `parent`, an error is raised. + +The type of a member access is the type of the `member` in the type of `parent`. + +At evaluation: + +. The `parent` is evaluated. +. The member access expression evaluates to the value of the `member` in the `parent` value. + +=== Function call expression +==== Syntax + +Rule: + +[listing] +---- +func_call: expr "(" [arguments] ")"` +arguments: argument ("," argument)* +argument: NAME "=" expr -> named_argument + | expr -> positional_argument +---- + +Name: + +`func_call: func "(" arguments ")"` + +`func` is resolved in the callable name group. + +All argument expressions are resolved in the value name group. + +==== Semantics + +At evaluation: + +. Each argument is evaluated from left to right. +. The behavior defined in the semantics for the <> referenced by `func` is + +===== Range semantics +The range operator is `..`. + +If `lhs` or `rhs` cannot be coerced to <>, an error is raised. diff --git a/docs/spec/12-execution.adoc b/docs/spec/12-execution.adoc new file mode 100644 index 0000000..473d40f --- /dev/null +++ b/docs/spec/12-execution.adoc @@ -0,0 +1,21 @@ +== Execution + +=== Control flow +A *branch* is a block of code which conditionally executes. + +> That is, whether or not that code executes depends on some expression. + +The list of statements which have branches is: + +* The <> +* The <> +* The <> +* The <> +TODO how does this definition handle assert/exit? + +=== Sleeping +The program may *sleep* until an absolute time called the *wakeup time*. + +While the current time is before the wakeup time, no statements may execute + +The program may also sleep for a time duration. This is the same as sleeping with a wakeup time of now, plus the specified duration. diff --git a/src/fpy/SPEC.md b/src/fpy/SPEC.md deleted file mode 100644 index 01a6dbc..0000000 --- a/src/fpy/SPEC.md +++ /dev/null @@ -1,279 +0,0 @@ -# The SPEC is currently work-in-progress - -# Types - -The following types are built into Fpy, and the developer can directly refer to them by name: -* Numeric types: `U8, U16, U32, U64, I8, I16, I32, I64, F32, F64` -* Boolean type: `bool` -* Time type: `Fw.Time` - -In addition, the developer can directly refer to any displayable type defined in FPP via its fully-qualified name. This includes user-defined structs, arrays and enums. - -There are some types which exist in Fpy but cannot be directly referenced by name by the developer. These are the *Int*, and *LiteralString* types. See [literals](#literals). - -## Structs -You can instantiate a new struct at runtime by calling its constructor. A struct's constructor is a function with the same name as the type, with arguments corresponding to the type and position of the struct's members. For example, a struct defined as: -``` -module Fw { - struct Example { - intValue: U8 - boolValue: bool - } -} -``` -can be constructed in Fpy like: -``` -Fw.Example(0, True) -``` - - -## Fields -Fields refer to either a member of a struct, or an element of an array. Field access uses Python-like syntax: `expr.member` reads a struct (or `Fw.TimeValue`) member and `expr[index]` reads an array element. These operations are only legal when the referenced type has a statically known layout. Because strings do not have a fixed size in memory, structs or arrays with string fields do not have a statically known layout. - -Accessing `Fw.TimeValue` produces synthetic members named `timeBase`, `timeContext`, `seconds`, and `useconds` with types `TimeBase` (enum), `FwTimeContextStoreType` (U8), `U32`, and `U32` respectively, matching the dictionary's `Fw.TimeValue` definition. `Fw.Time` is an alias for `Fw.TimeValue`. - -Array indices are coerced to `I64` before use. If the index is a compile-time constant the compiler emits an error when it falls outside `[0, length)`. Otherwise the generated bytecode performs a runtime bounds check and terminates the sequence with `DirectiveErrorCode.ARRAY_OUT_OF_BOUNDS` if it fails. - -# Literals -The following literals are supported by Fpy: -* Integer literals: `123`, `-456_879` -* Float literals: `0.123`, `1e-5` -* String literals: `"hello world"`, `'example string'` -* Boolean literals: `True` and `False` - -## Integer literals -Integer literals are strings matching: -``` -DEC_NUMBER: "1".."9" ("_"? "0".."9" )* - | "0" ("_"? "0" )* /(?![1-9])/ -``` - -The first rule of this syntax allows for integers without leading zeroes, separated by underscores. So this is okay: -``` -123_456 -``` -but this is not: -``` -0123_456 -``` - -The second rule allows you to write any number of zeroes, separated by underscores: -``` -00_000_0 -``` - -Integer literals have type *Int*, which is not directly referenceable by the user. The *Int* type supports integers of arbitrary size. - -## Float literals -Float literals are strings matching: -``` -FLOAT_NUMBER: _SPECIAL_DEC _EXP | DECIMAL _EXP? -``` -where `_SPECIAL_DEC`, `_EXP` and `DECIMAL` are defined as: - -``` -_SPECIAL_DEC: "0".."9" ("_"? "0".."9")* -_EXP: ("e"|"E") ["+" | "-"] _SPECIAL_DEC -DECIMAL: "." _SPECIAL_DEC | _SPECIAL_DEC "." _SPECIAL_DEC -``` - -A `FLOAT_NUMBER` can be any string of digits suffixed with an exponent, like these: -``` -1e-5 -100_200e10 -``` - -or it can be a `DECIMAL` optionally suffixed by an exponent, like these: -``` -2.123 -100.5e+10 -``` - -Float literals have type `F64`. - -## String literals -String literals are strings matching: -``` -STRING: /("(?!"").*?(? F64`: computes the natural logarithm of the operand using `FloatLogDirective` and leaves the `F64` result on the stack. -* `sleep(seconds: U32 = 0, microseconds: U32 = 0)`: waits for the specified relative duration (the assembler emits `WaitRelDirective`). -* `sleep_until(wakeup_time: Fw.TimeValue)`: waits until the supplied absolute time using `WaitAbsDirective`. -* `now() -> Fw.TimeValue`: pushes the current time via `PushTimeDirective`. -* `rand() -> U32`: pushes the next random number via `PushRandDirective`. -* `randf() -> F64`: pushes the next random number via `PushRandDirective`, converts it to `F64`, and divides by `2**32` to produce a value in the half-open range `[0.0, 1.0)`. -* `set_seed(seed: U32)`: seeds the random number generator via `SetSeedDirective`. -* `time(timestamp: String, timeBase: TimeBase = TimeBase.TB_NONE, timeContext: U8 = 0) -> Fw.TimeValue`: parses an ISO 8601 timestamp string (e.g., `"2025-12-19T14:30:00Z"` or `"2025-12-19T14:30:00.123456Z"`) at compile time and returns an `Fw.TimeValue` with the specified `timeBase` and `timeContext`. The timestamp must be in UTC with a `Z` suffix. -* `iabs(value: I64) -> I64`: returns the absolute value of a signed 64-bit integer by emitting an `IntAbsDirective`. The absolute value of `I64` min (`-2**63`) wraps back to `I64` min rather than trapping, matching libm's `llabs`. -* `fabs(value: F64) -> F64`: returns the absolute value of a 64-bit float by emitting a `FloatAbsDirective`. -* `log(message: String, severity: Fw.LogSeverity = Fw.LogSeverity.ACTIVITY_HI)`: pushes the severity, message, and message size onto the stack, then emits a `PopEventDirective`. Both arguments must be compile-time constants. - -## Type constructors -Structs, arrays, and `Fw.TimeValue` expose constructors whose callable name is the fully qualified type name. Their arguments correspond to the members in declaration order (struct fields by name, array elements as `e0`, `e1`, ..., and `Fw.TimeValue` with `timeBase: TimeBase`, `timeContext: U8`, `seconds: U32`, `useconds: U32`). A constructor call serializes the provided values into a new instance of that type. `Fw.Time` is an alias for `Fw.TimeValue`, and `Fw.TimeInterval` is an alias for `Fw.TimeIntervalValue`. - -## Numeric casts -Each concrete numeric type provides a callable whose name matches the type (for example `U16(value)` or `F64(value)`). Casts accept exactly one numeric argument. Unlike implicit coercion, casts always force the operand into the target type even when this requires narrowing; range checks are suppressed and the value is truncated or rounded if necessary. See [Casting](#casting) for details. - -## User-defined functions -A function definition introduces a new callable into scope. Function definitions must appear at the top level of the sequence; a function definition nested inside another function, a loop body, or a conditional branch is a compile-time error. The syntax is: -``` -def name(param_0: Type0, param_1: Type1 = default_value, ...) [-> ReturnType]: - body -``` - -### Parameters -Each parameter declaration consists of a name followed by a colon and a type annotation. A parameter may optionally include a default value, written as `= expr` after the type annotation. Default value expressions must be constant expressions: literals, enum constants, or type constructors whose arguments are themselves constant expressions. Expressions referencing telemetry channels, variables, or function calls are not constant and produce a compile-time error when used as defaults. - -Arguments may be passed by position or by name. Positional arguments are bound to parameters left-to-right. Named arguments use the syntax `name=expr` and bind the value of `expr` to the parameter with the matching name. All positional arguments must precede all named arguments. A parameter may not be bound more than once; supplying both a positional argument and a named argument for the same parameter is a compile-time error. If fewer arguments are supplied than parameters, the remaining parameters must have default values; those defaults are evaluated and bound. Supplying more positional arguments than parameters, or naming a parameter that does not exist, is a compile-time error. - -### Return type -The return type annotation `-> Type` is optional. When present, every control-flow path through the function body must terminate with a `return expr` statement where `expr` has a type coercible to `Type`. When absent, the function does not produce a value; `return` statements in such functions must not include an expression, and the call expression has no usable result. - -### Scope -A function body introduces a new scope. Within this scope the following names are visible: -1. Parameters declared in the function signature. -2. Local variables declared within the function body. -3. Top-level variables declared before the call site of the function (not the definition site). -4. All dictionary objects: commands, telemetry channels, parameters, enum constants, and types. -5. All user-defined functions, including functions defined after the current function (forward references are permitted). - -Assignments to top-level variables within a function body modify the original variable. Assignments to parameters or local variables do not affect any outer scope. - -# Type conversion - -Type conversion is the process of converting an expression from one type to another. It can either be implicit, in which case it is called coercion, or explicit, in which case it is called casting. - -## Coercion -Coercion happens when an expression of type *A* is used in a syntactic element which requires an expression of type *B*. For example, functions, operators and variable assignments all require specific input types, so type coercion happens in each of these. -In general, the rule of thumb is that coercion is allowed if the destination type can represent all possible values of the source type, with some exceptions. The following rules determine when type coercion can be performed: - -1. If the source and destination types are identical, no coercion is performed. -2. *LiteralString* values may be coerced into any FPP string type. No other string expression can be coerced. -3. Otherwise both source and destination must be numeric (`NumericalValue`). Numeric coercions obey these constraints: - * Floats never coerce to integers. - * Integers may always coerce to floats. - * Float-to-float coercions require a destination bit width greater than or equal to the source width. - * Integer-to-integer coercions require matching signedness and a destination bit width greater than or equal to the source width. - * Arbitrary-precision types (`Int`/`Float`) may coerce to any finite-width numeric type. -If no rule matches, the compiler raises an error. - -Compile-time constant floats (including literals and constant-folded expressions) can only be narrowed into a smaller floating-point type when the value lies inside the destination’s representable range. When the value fits, the compiler rounds it to the nearest representable floating-point number; otherwise compilation fails with an out-of-range error. - -## Casting -Each finite-bitwidth numeric type exposes an explicit cast with the same name as the type, e.g. `U32(value)` or `F64(value)`. Casts accept any numeric expression and bypass the implicit-coercion restrictions above: the operand is forced to the target type even when that entails narrowing, and compile-time range checks are suppressed. No casts exist for structs, arrays, enums, strings, or `Fw.Time`. - - -# Operators - -Fpy supports the following operators: -* Basic arithmetic: `+, -, *, /` -* Modulo: `%` -* Exponentiation: `**` -* Floor division: `//` -* Boolean: `and, or, not` -* Comparison: `<, >, <=, >=, ==, !=` - -Each time an operator is used, an intermediate type must be picked and both args must be converted to that type. - -## Behavior of operators -All operators share the following rules: - -1. The left operand is evaluated first, then the right operand. Boolean `and`/`or` short-circuit, so the right operand is skipped when the result is already known. -2. Each operand is coerced to the operator’s intermediate type (see [Intermediate Types](#intermediate-types)). If no valid intermediate type exists, compilation fails. - -The subsections below describe behaviors that differ from the general rules. - -### Numeric arithmetic (`+`, `-`, `*`) -These operators require numeric operands and produce a result in the chosen intermediate type. Addition, subtraction, and multiplication differ only in which arithmetic operation they perform. Integer overflow wraps according to the destination type when the result is ultimately stored, and floating-point operations follow IEEE-754 behavior. - -### True division (`/`) -Both operands are promoted to `F64`, and the result is always an `F64`. This means you must explicitly cast the result to store it in an integer type. - -### Floor division (`//`) -With integer operands, `//` performs truncating division using the signed or unsigned divide directive. If either operand is a float, the compiler divides in `F64`, converts the quotient to a signed 64-bit integer (which truncates toward zero), and converts back to `F64`, so floating-point floor division also truncates toward zero. - -### Modulo (`%`) -Modulo works for numeric operands. Signed operands use the signed modulo directive, unsigned operands use the unsigned directive, and floats use floating-point modulo. For signed integers the remainder has the same sign as the dividend. - -### Exponentiation (`**`) -Both operands are coerced to `F64`, the exponentiation happens in floating point, and the result type is `F64`. - -### Boolean operators (`and`, `or`, `not`) -Operands must be `bool`. `not` negates a single operand. `and` evaluates the left operand first and only evaluates the right operand when the left operand is `True`. Conversely, `or` skips the right operand when the left operand is `True`. The result of every boolean operator is `bool`. - -### Inequalities (`<`, `<=`, `>`, `>=`) -Inequalities require numeric operands. Each operand is coerced to the intermediate type, the comparison runs in that type, and the result is `bool`. - -### Equality (`==`, `!=`) -If both operands are numeric, equality uses the same intermediate-type rules as arithmetic operators. Otherwise both operands must have the exact same concrete type (struct, array, enum, or `Fw.Time`). The compiler compares their serialized bytes. Strings cannot be compared. - -## Intermediate types - -Intermediate types are picked via the following rules: - -1. The intermediate type of Boolean operators is always `bool`. -2. The intermediate type of `==` and `!=` may be any type, so long as the left and right hand sides are the same type. If both are numeric then continue. -3. If either argument is non-numeric, raise an error. -4. If the operator is `/` or `**`, the intermediate type is always `F64`. -5. If either argument is a float, the intermediate type is `F64`. -6. If either argument is an unsigned integer, the intermediate type is `U64`. -7. Otherwise, the intermediate type is `I64`. - -If the expressions given to the operator are not of the intermediate type, type coercion rules are applied. - -## Result type - -The result type is the type of the value produced by the operator. -1. For numeric operators, the result type is the intermediate type. -2. For boolean and comparison operators, the result type is `bool`. - -Normal type coercion rules apply to the result, of course. Once the operator has produced a value, it may be coerced into some other type depending on context. - -# Loops - -## Range expressions - -The `lower .. upper` operator produces a `RangeValue`. Both bounds are coerced to `I64`. Range expressions are only meaningful as the right-hand side of a `for` loop, and both bounds are evaluated exactly once. - -## For loops - -``` -for in .. : - -``` - -* `` and `` are evaluated before the loop starts and stored in hidden variables. -* The loop variable is assigned ``, coerced to `I64`. If the name was not previously declared, the compiler declares it with type `I64`; otherwise the existing variable must already be of that type. -* The body runs while the loop variable is strictly less than ``. After each iteration the compiler inserts `var = var + 1`. Modifying `var` manually inside the body affects the next iteration in addition to this implicit increment. -* `break` and `continue` statements are only legal inside the loop body and behave as in C: `break` exits the loop, `continue` skips to the implicit increment/check sequence. - -## While loops - -``` -while : - -``` - -The condition is coerced to `bool` and re-evaluated before every iteration. `break` and `continue` are legal only within the loop body. diff --git a/src/fpy/codegen_llvm.py b/src/fpy/codegen_llvm.py index 6dd83f7..9779920 100644 --- a/src/fpy/codegen_llvm.py +++ b/src/fpy/codegen_llvm.py @@ -51,22 +51,28 @@ ERROR_CODE_TYPE = ErrorCodeType.llvm_type -# Host import that ends the whole sequence +# Host imports that end the whole sequence. They are two deliberately +# separate channels (mirroring the C++ sequencer's SequenceExitedWithError +# event vs DirectiveError): fpy_exit carries a *user-chosen* code (exit(), +# assert), fpy_fault carries a DirectiveErrorCode raised by a runtime check +# (division by zero, arithmetic overflow). Keeping them apart means a +# sequence calling exit(10) can never impersonate a DOMAIN_ERROR fault. HOST_EXIT_FUNC_NAME = "fpy_exit" +HOST_FAULT_FUNC_NAME = "fpy_fault" # TODO this adhoc emit and declare stuff is not super nice. should probably # clean it up, have a list of the expected host module signature funcs, declare them # all at the start. -def _declare_host_exit(module: ir.Module) -> ir.Function: - """Get-or-declare the fpy_exit host import on *module*.""" - existing = module.globals.get(HOST_EXIT_FUNC_NAME) +def _declare_host_noreturn(module: ir.Module, name: str) -> ir.Function: + """Get-or-declare a void(i32) noreturn host import on *module*.""" + existing = module.globals.get(name) if existing is not None: return existing fn = ir.Function( module, ir.FunctionType(ir.VoidType(), [ERROR_CODE_TYPE]), - name=HOST_EXIT_FUNC_NAME, + name=name, ) # It never returns to wasm: the host unwinds the interpreter. fn.attributes.add("noreturn") @@ -74,8 +80,16 @@ def _declare_host_exit(module: ir.Module) -> ir.Function: def emit_host_exit(builder: ir.IRBuilder, code: ir.Value) -> None: - """Emit a call to fpy_exit(code) and terminate the current block.""" - builder.call(_declare_host_exit(builder.module), [code]) + """Emit a call to fpy_exit(code) -- the user-exit channel -- and + terminate the current block.""" + builder.call(_declare_host_noreturn(builder.module, HOST_EXIT_FUNC_NAME), [code]) + builder.unreachable() + + +def emit_host_fault(builder: ir.IRBuilder, code: ir.Value) -> None: + """Emit a call to fpy_fault(code) -- the runtime-fault channel -- and + terminate the current block.""" + builder.call(_declare_host_noreturn(builder.module, HOST_FAULT_FUNC_NAME), [code]) builder.unreachable() @@ -150,6 +164,11 @@ def emit_AstBinaryOp(self, node: AstBinaryOp, state: CompileState) -> ir.Value: assert op in COMPARISON_OPS, op if is_float: + if op == BinaryStackOp.NOT_EQUAL: + # IEEE 754 defines != as the negation of ==, so it is true + # when either operand is NaN: that's fcmp une (unordered); + # fcmp_ordered would emit `one`, which is false on NaN. + return b.fcmp_unordered(op, lhs, rhs) return b.fcmp_ordered(op, lhs, rhs) # Enums and bools lower to integers too, so any integer-typed value # (not just numeric types) compares with icmp; aggregates don't. @@ -164,6 +183,18 @@ def emit_AstBinaryOp(self, node: AstBinaryOp, state: CompileState) -> ir.Value: f"'{intermediate_type.display_name}' yet" ) + def _emit_halt_if(self, cond: ir.Value, code: DirectiveErrorCode) -> None: + """Guard an operation: end the whole sequence with the runtime fault + *code* when cond holds, otherwise fall through and continue lowering. + Faults go through fpy_fault, not fpy_exit: they are not user exits.""" + b = self.builder + fail_block = b.append_basic_block("arith_fail") + ok_block = b.append_basic_block("arith_ok") + b.cbranch(cond, fail_block, ok_block) + b.position_at_end(fail_block) + emit_host_fault(b, ir.Constant(ERROR_CODE_TYPE, code.value)) + b.position_at_end(ok_block) + def _emit_floor_divide( self, lhs: ir.Value, rhs: ir.Value, is_float: bool, is_signed: bool ) -> ir.Value: @@ -182,12 +213,29 @@ def _emit_floor_divide( quotient = b.fdiv(lhs, rhs) floor_fn = b.module.declare_intrinsic("llvm.floor", [quotient.type]) return b.call(floor_fn, [quotient]) + + zero = ir.Constant(lhs.type, 0) + # udiv/sdiv are immediate UB on a zero divisor in LLVM; the sequence + # must instead end with the same error the VM's handle_udiv/handle_sdiv + # return. + self._emit_halt_if( + b.icmp_unsigned("==", rhs, zero), DirectiveErrorCode.DOMAIN_ERROR + ) if not is_signed: # Unsigned operands are non-negative, so the exact quotient is too; # there's nothing below zero to floor toward, so udiv (which # truncates) already gives the floored result. return b.udiv(lhs, rhs) + # sdiv MIN,-1 is UB as well: the mathematical quotient 2^(n-1) is not + # representable, so the sequence ends with an overflow error. + int_min = ir.Constant(lhs.type, -(1 << (lhs.type.width - 1))) + minus_one = ir.Constant(lhs.type, -1) + overflow = b.and_( + b.icmp_signed("==", lhs, int_min), b.icmp_signed("==", rhs, minus_one) + ) + self._emit_halt_if(overflow, DirectiveErrorCode.ARITHMETIC_OVERFLOW) + # Signed integers have no floor instruction: sdiv truncates toward zero. # Truncation and floor agree except when the exact quotient is negative # and non-integer -- i.e. the operands have opposite signs (negative @@ -221,11 +269,18 @@ def _emit_modulo( lowers to an fmod libcall on wasm, hence the imported env.fmod.) """ b = self.builder + zero = ir.Constant(lhs.type, 0) + if not is_float: + # urem/srem are immediate UB on a zero divisor in LLVM; the + # sequence must instead end with the same error the VM's + # handle_umod/handle_smod return. + self._emit_halt_if( + b.icmp_unsigned("==", rhs, zero), DirectiveErrorCode.DOMAIN_ERROR + ) if not is_float and not is_signed: # Unsigned operands are non-negative, so floored == truncated. return b.urem(lhs, rhs) - zero = ir.Constant(lhs.type, 0) if is_float: rem = b.frem(lhs, rhs) nonzero = b.fcmp_ordered("!=", rem, zero) @@ -234,6 +289,16 @@ def _emit_modulo( ) corrected = b.fadd(rem, rhs) else: + # srem MIN,-1 is UB (and errors in the VM and in Rust) even + # though the remainder itself would be 0: it halts exactly like + # MIN // -1 does. + int_min = ir.Constant(lhs.type, -(1 << (lhs.type.width - 1))) + minus_one = ir.Constant(lhs.type, -1) + overflow = b.and_( + b.icmp_signed("==", lhs, int_min), + b.icmp_signed("==", rhs, minus_one), + ) + self._emit_halt_if(overflow, DirectiveErrorCode.ARITHMETIC_OVERFLOW) rem = b.srem(lhs, rhs) nonzero = b.icmp_signed("!=", rem, zero) # rem and rhs have differing signs iff their xor is negative. diff --git a/src/fpy/model.py b/src/fpy/model.py index 343d490..6c5836b 100644 --- a/src/fpy/model.py +++ b/src/fpy/model.py @@ -1066,6 +1066,11 @@ def handle_sdiv(self, dir: SignedIntDivideDirective): if rhs == 0: return DirectiveErrorCode.DOMAIN_ERROR + if lhs == MIN_INT64 and rhs == -1: + # the mathematical quotient 2^63 is not representable; without + # this check push() would silently truncate it back to MIN_INT64 + return DirectiveErrorCode.ARITHMETIC_OVERFLOW + result = lhs // rhs self.push(result) @@ -1119,6 +1124,10 @@ def handle_smod(self, dir: SignedModuloDirective): lhs = self.pop(signed=True) if rhs == 0: return DirectiveErrorCode.DOMAIN_ERROR + if lhs == MIN_INT64 and rhs == -1: + # matches sdiv (and Rust): the remainder itself (0) is fine, but + # the operation is UB in C++/LLVM, so it errors like MIN // -1 + return DirectiveErrorCode.ARITHMETIC_OVERFLOW self.push(lhs % rhs, signed=True) return None @@ -1137,9 +1146,19 @@ def handle_fmod(self, dir: FloatModuloDirective): return DirectiveErrorCode.STACK_UNDERFLOW rhs = self.pop(type=float) lhs = self.pop(type=float) - if rhs == 0.0: - return DirectiveErrorCode.DOMAIN_ERROR - self.push(lhs % rhs) + # IEEE semantics (and Rust's/C#'s): float % never errors. x % 0.0, + # inf % y, and NaN operands give NaN. Otherwise the result is floored + # fmod (sign of the divisor), computed exactly like the spec and the + # LLVM backend (fmod, then one addition of the divisor on sign + # mismatch). CPython raises on % 0.0 instead; fpy deliberately follows + # IEEE (see MATH_COMPARISON.md). + if rhs == 0.0 or math.isnan(lhs) or math.isnan(rhs) or math.isinf(lhs): + self.push(float("nan")) + return None + result = math.fmod(lhs, rhs) + if result != 0.0 and (result < 0.0) != (rhs < 0.0): + result += rhs + self.push(result) return None def handle_fpow(self, dir: FloatExponentDirective): diff --git a/src/fpy/semantics.py b/src/fpy/semantics.py index 4b24b95..d35c10a 100644 --- a/src/fpy/semantics.py +++ b/src/fpy/semantics.py @@ -1648,6 +1648,18 @@ def pick_intermediate_type( if not all(t.is_numerical for t in arg_types): return None + # negation is undefined for unsigned integers (as in Rust): the result + # is negative for every nonzero operand, which no unsigned type can + # represent. NOTE: the arity check is what distinguishes negation from + # subtraction -- ops are str-valued and both are "-" (and the AST + # carries plain strings, so an enum identity check won't work either) + if ( + len(arg_types) == 1 + and op == UnaryStackOp.NEGATE + and arg_types[0] in UNSIGNED_INTEGER_TYPES + ): + return None + # division and exponentiation always operate over floats if op in (BinaryStackOp.DIVIDE, BinaryStackOp.EXPONENT): if all(t in ARBITRARY_PRECISION_TYPES for t in arg_types): diff --git a/src/fpy/test_helpers.py b/src/fpy/test_helpers.py index 5afebb5..0a50d04 100644 --- a/src/fpy/test_helpers.py +++ b/src/fpy/test_helpers.py @@ -101,13 +101,16 @@ def compile_seq_wasm( return wasm -def run_seq_wasm( +def run_seq_wasm_outcome( seq: str, ground_binary_dir: str = None, import_search_dirs: list[str] | None = None -) -> int: - """Compile *seq* to wasm and run it, returning fpy_main's error code. - - Runs the compiled module through the NASA spacewasm interpreter (the - on-board target runtime) via the runner harness built by conftest.""" +) -> tuple[str, int]: + """Compile *seq* to wasm and run it, returning (channel, code). + + channel is "exit" (user-exit: exit(), assert, or fpy_main returning + normally) or "fault" (a runtime fault raised by a compiler-emitted guard, + carrying a DirectiveErrorCode). Runs the compiled module through the NASA + spacewasm interpreter (the on-board target runtime) via the runner harness + built by conftest.""" import subprocess assert ( @@ -131,7 +134,27 @@ def run_seq_wasm( f"spacewasm runner faulted (exit {result.returncode}): " f"{result.stderr.strip()}" ) - return int(result.stdout.strip()) + channel, code = result.stdout.strip().split() + assert channel in ("exit", "fault"), result.stdout + return channel, int(code) + + +def run_seq_wasm( + seq: str, ground_binary_dir: str = None, import_search_dirs: list[str] | None = None +) -> int: + """Compile *seq* to wasm and run it, returning the user-exit code. + + Raises if the sequence ends with a runtime *fault* instead (division by + zero, arithmetic overflow, ...): callers of this helper expect user-exit + semantics; fault-expecting tests go through assert_run_failure.""" + channel, code = run_seq_wasm_outcome( + seq, ground_binary_dir, import_search_dirs=import_search_dirs + ) + if channel != "exit": + raise RuntimeError( + f"wasm sequence ended with runtime fault {DirectiveErrorCode(code)}" + ) + return code def lookup_type(fprime_test_api, type_name: str): @@ -380,22 +403,46 @@ def assert_run_failure( error_code is not None or validation_error ), "Must specify either error_code or validation_error" + # The expected failure's channel: a DirectiveErrorCode other than + # EXIT_WITH_ERROR is a runtime fault (guards); EXIT_WITH_ERROR (a bare + # assert) and raw ints (exit(n)) come through the user-exit channel. + # The channels must not cross-match: exit(10) does not satisfy an + # expected DOMAIN_ERROR even though DOMAIN_ERROR's value is 10. + expect_fault = ( + isinstance(error_code, DirectiveErrorCode) + and error_code != DirectiveErrorCode.EXIT_WITH_ERROR + ) + # ...except that the bytecode ISA has no fault-raising directive, so the + # compiler lowers ITS OWN runtime checks (array bounds, assert_cmd_success) + # to `PushVal(code); Exit` -- on the VM these semantically-fault codes ride + # the exit channel by construction. TODO(upstream): give the FpySequencer + # ISA a fault directive so these stop being spoofable via exit(). + COMPILED_IN_CHECK_CODES = { + DirectiveErrorCode.ARRAY_OUT_OF_BOUNDS, + DirectiveErrorCode.CMD_FAIL, + } + if USE_WASM: - # The wasm backend has no separate validation step or VM-internal - # faults: a failed sequence is one whose entry point returns nonzero. - code = run_seq_wasm( + # The wasm backend has no separate validation step; a failed sequence + # either exits nonzero (user channel) or raises a runtime fault. + channel, code = run_seq_wasm_outcome( seq, ground_binary_dir=ground_binary_dir, import_search_dirs=import_search_dirs, ) - if code == DirectiveErrorCode.NO_ERROR.value: + if (channel, code) == ("exit", DirectiveErrorCode.NO_ERROR.value): raise RuntimeError("wasm sequence succeeded") if error_code is not None: - if ( - isinstance(error_code, DirectiveErrorCode) and code != error_code.value - ) or (isinstance(error_code, int) and code != error_code): + want_channel = "fault" if expect_fault else "exit" + want_code = ( + error_code.value + if isinstance(error_code, DirectiveErrorCode) + else error_code + ) + if (channel, code) != (want_channel, want_code): raise RuntimeError( - f"wasm sequence returned {code}, expected {error_code}" + f"wasm sequence ended with {channel} {code}, " + f"expected {want_channel} {want_code} ({error_code})" ) return @@ -452,16 +499,28 @@ def assert_run_failure( if validation_error: raise RuntimeError("Expected ValidationError, got", type(e).__name__, e) - # The failure surfaces as either a DirectiveErrorCode trap or a raw exit - # code int; the expected value may likewise be either. Compare by integer - # value so e.g. an exit code of 7 matches DirectiveErrorCode.EXIT_WITH_ERROR. - def _as_int(v): - return v.value if isinstance(v, DirectiveErrorCode) else v - - if len(e.args) == 1 and _as_int(e.args[0]) != _as_int(error_code): - raise RuntimeError( - "run_seq failed with error", e.args[0], "expected", error_code - ) + # The failure's channel is encoded in the arg type: a runtime fault + # surfaces as a DirectiveErrorCode trap, a user exit as a raw int. + # The channels must not cross-match (see expect_fault above), except + # for the compiled-in checks that the bytecode ISA forces through the + # exit channel. + if len(e.args) == 1: + got = e.args[0] + if expect_fault: + ok = isinstance(got, DirectiveErrorCode) and got == error_code + if error_code in COMPILED_IN_CHECK_CODES: + ok = ok or got == error_code.value + else: + want = ( + error_code.value + if isinstance(error_code, DirectiveErrorCode) + else error_code + ) + ok = not isinstance(got, DirectiveErrorCode) and got == want + if not ok: + raise RuntimeError( + "run_seq failed with error", got, "expected", error_code + ) print(e) return diff --git a/test/fpy/test_arithmetic.py b/test/fpy/test_arithmetic.py index 52af421..719cbc9 100644 --- a/test/fpy/test_arithmetic.py +++ b/test/fpy/test_arithmetic.py @@ -1,8 +1,13 @@ import pytest +from fpy.model import DirectiveErrorCode from fpy.types import U32 -from fpy.test_helpers import assert_compile_failure, assert_run_success +from fpy.test_helpers import ( + assert_compile_failure, + assert_run_failure, + assert_run_success, +) class TestConstantFolding: @@ -617,3 +622,159 @@ def test_int_floor_div_negative_divisor(self, fprime_test_api): assert result == -4 """ assert_run_success(fprime_test_api, seq) + + +class TestIntDivisionGuards: + """Runtime integer division/modulo must end the sequence instead of + hitting undefined behavior: zero divisors are a DOMAIN_ERROR (matching + the VM's handle_udiv/sdiv/umod/smod), and I64_MIN // -1 is an + ARITHMETIC_OVERFLOW (its quotient 2^63 is unrepresentable). Operands are + variables so the const folder can't resolve them at compile time.""" + + def test_signed_floor_div_by_zero_halts(self, fprime_test_api): + seq = """ +a: I64 = 1 +b: I64 = 0 +result: I64 = a // b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.DOMAIN_ERROR) + + def test_unsigned_floor_div_by_zero_halts(self, fprime_test_api): + seq = """ +a: U64 = 1 +b: U64 = 0 +result: U64 = a // b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.DOMAIN_ERROR) + + def test_signed_mod_by_zero_halts(self, fprime_test_api): + seq = """ +a: I64 = 1 +b: I64 = 0 +result: I64 = a % b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.DOMAIN_ERROR) + + def test_unsigned_mod_by_zero_halts(self, fprime_test_api): + seq = """ +a: U64 = 1 +b: U64 = 0 +result: U64 = a % b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.DOMAIN_ERROR) + + def test_int_min_floor_div_minus_one_halts(self, fprime_test_api): + seq = """ +a: I64 = -9223372036854775808 +b: I64 = -1 +result: I64 = a // b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.ARITHMETIC_OVERFLOW) + + def test_int_min_mod_minus_one_halts(self, fprime_test_api): + """I64_MIN % -1 halts like I64_MIN // -1 (Rust's rule): the + mathematical remainder would be 0, but the operation is UB in + C++/LLVM and // and % halt on exactly the same inputs.""" + seq = """ +a: I64 = -9223372036854775808 +b: I64 = -1 +result: I64 = a % b +""" + assert_run_failure(fprime_test_api, seq, DirectiveErrorCode.ARITHMETIC_OVERFLOW) + + +class TestUnaryMinusUnsigned: + """Unary minus is undefined for unsigned integer types (compile error, + as in Rust): the result is negative for every nonzero operand, which no + unsigned type can represent.""" + + def test_negate_unsigned_var_is_compile_error(self, fprime_test_api): + seq = """ +a: U32 = 5 +b: I64 = -a +""" + assert_compile_failure(fprime_test_api, seq, match="undefined") + + def test_negate_unsigned_const_is_compile_error(self, fprime_test_api): + seq = """ +a: I64 = -U8(5) +""" + assert_compile_failure(fprime_test_api, seq, match="undefined") + + def test_negate_signed_and_float_still_work(self, fprime_test_api): + seq = """ +a: I32 = 5 +b: F64 = 2.5 +assert -a == -5 +assert -b == -2.5 +""" + assert_run_success(fprime_test_api, seq) + + +class TestNaNComparisons: + """IEEE 754 comparisons with NaN: every comparison is false, except != + which is the negation of == and hence true. The NaN is produced at + runtime (0.0/0.0 of variables) so nothing is const-folded.""" + + def test_nan_neq_nan_is_true(self, fprime_test_api): + seq = """ +zero: F64 = 0.0 +nan: F64 = zero / zero +assert nan != nan +""" + assert_run_success(fprime_test_api, seq) + + def test_nan_eq_nan_is_false(self, fprime_test_api): + seq = """ +zero: F64 = 0.0 +nan: F64 = zero / zero +assert not (nan == nan) +""" + assert_run_success(fprime_test_api, seq) + + def test_nan_ordered_comparisons_are_false(self, fprime_test_api): + seq = """ +zero: F64 = 0.0 +nan: F64 = zero / zero +assert not (nan < 1.0) +assert not (nan <= 1.0) +assert not (nan > 1.0) +assert not (nan >= 1.0) +""" + assert_run_success(fprime_test_api, seq) + + +class TestFloatModIEEE: + """Float % follows IEEE (and Rust/C#): it never halts. x % 0.0 and + inf % y are NaN. (CPython raises ZeroDivisionError instead; fpy + deliberately follows IEEE -- see MATH_COMPARISON.md.) Operands are + variables so nothing is const-folded.""" + + def test_float_mod_by_zero_is_nan(self, fprime_test_api): + seq = """ +a: F64 = 1.0 +b: F64 = 0.0 +c: F64 = a % b +assert c != c +""" + assert_run_success(fprime_test_api, seq) + + def test_inf_mod_is_nan(self, fprime_test_api): + seq = """ +one: F64 = 1.0 +zero: F64 = 0.0 +inf: F64 = one / zero +c: F64 = inf % 2.0 +assert c != c +""" + assert_run_success(fprime_test_api, seq) + + def test_finite_mod_inf_is_identity(self, fprime_test_api): + seq = """ +one: F64 = 1.0 +zero: F64 = 0.0 +inf: F64 = one / zero +c: F64 = -5.0 % inf +assert c == inf +""" + assert_run_success(fprime_test_api, seq) diff --git a/test/fpy/test_assert.py b/test/fpy/test_assert.py index 57102b5..38a5c74 100644 --- a/test/fpy/test_assert.py +++ b/test/fpy/test_assert.py @@ -43,3 +43,19 @@ def test_assert_wrong_exit_code_type(self, fprime_test_api): """ assert_compile_failure(fprime_test_api, seq) + + def test_exit_code_does_not_impersonate_fault(self, fprime_test_api): + """User exits and runtime faults are separate channels: exit(10) + matches the raw code 10, but must NOT satisfy an expected + DOMAIN_ERROR fault even though DOMAIN_ERROR's value is also 10.""" + if fprime_test_api is not None: + return # GDS mode reports failures via events, not channels + assert_run_failure(fprime_test_api, "exit(10)", 10) + try: + assert_run_failure( + fprime_test_api, "exit(10)", DirectiveErrorCode.DOMAIN_ERROR + ) + except RuntimeError: + pass + else: + raise AssertionError("exit(10) was accepted as a DOMAIN_ERROR fault") diff --git a/test/fpy/test_model.py b/test/fpy/test_model.py index 8770c86..be2f738 100644 --- a/test/fpy/test_model.py +++ b/test/fpy/test_model.py @@ -618,14 +618,16 @@ def test_signed_div_by_zero(self): assert result == DirectiveErrorCode.DOMAIN_ERROR def test_signed_div_min_by_neg_one(self): - """Test sdiv MIN_INT64 / -1 special case (overflow to MIN_INT64).""" + """sdiv MIN_INT64 / -1: the mathematical quotient 2^63 is not + representable in I64, so the directive errors out instead of + silently wrapping. (The C++ op_sdiv computes `lhs / rhs`, which is + undefined behavior for exactly this input and needs the same + guard.)""" model = FpySequencerModel() model.push(MIN_INT64) model.push(-1) result = model.dispatch(SignedIntDivideDirective()) - assert result == DirectiveErrorCode.NO_ERROR or result is None - ret = model.pop() - assert ret == MIN_INT64 + assert result == DirectiveErrorCode.ARITHMETIC_OVERFLOW def test_float_add_stack_underflow(self): """Test fadd with insufficient stack.""" @@ -676,6 +678,34 @@ def test_float_mod_stack_underflow(self): result = model.dispatch(FloatModuloDirective()) assert result == DirectiveErrorCode.STACK_UNDERFLOW + def test_float_mod_by_zero_is_nan(self): + """fmod never errors (IEEE, matching Rust/C# and the LLVM backend): + x % 0.0 is NaN, not a DOMAIN_ERROR.""" + model = FpySequencerModel() + model.push(1.0) + model.push(0.0) + result = model.dispatch(FloatModuloDirective()) + assert result is None or result == DirectiveErrorCode.NO_ERROR + assert math.isnan(model.pop(type=float)) + + def test_float_mod_inf_dividend_is_nan(self): + """inf % y is NaN per IEEE.""" + model = FpySequencerModel() + model.push(float("inf")) + model.push(2.0) + result = model.dispatch(FloatModuloDirective()) + assert result is None or result == DirectiveErrorCode.NO_ERROR + assert math.isnan(model.pop(type=float)) + + def test_float_mod_is_floored(self): + """The result takes the divisor's sign (Python-style floored mod).""" + model = FpySequencerModel() + model.push(-7.5) + model.push(2.0) + result = model.dispatch(FloatModuloDirective()) + assert result is None or result == DirectiveErrorCode.NO_ERROR + assert model.pop(type=float) == 0.5 + def test_float_pow_stack_underflow(self): """Test fpow with insufficient stack.""" model = FpySequencerModel() diff --git a/test/spacewasm_runner/src/main.rs b/test/spacewasm_runner/src/main.rs index 9ab4ae7..c7e7b99 100644 --- a/test/spacewasm_runner/src/main.rs +++ b/test/spacewasm_runner/src/main.rs @@ -3,17 +3,21 @@ //! //! Usage: `fpy-spacewasm-runner [entry-name]` //! -//! On success it prints the sequence's i32 error code as a single decimal line -//! to stdout and exits 0. The code comes from `env.fpy_exit` when the sequence -//! calls exit() or fails an assert, or from `fpy_main`'s return value when the -//! sequence falls off its end (0 == success). The caller reads the printed -//! code; the process exit status only distinguishes "ran cleanly" (0) from -//! "harness/runtime fault" (2), so a nonzero error code is not conflated with a -//! trap. +//! On success it prints one line to stdout, ` `, and exits 0: +//! +//! * `exit ` -- the user-exit channel: `env.fpy_exit` (exit() or a +//! failing assert), or `fpy_main` returning normally (code 0). +//! * `fault ` -- the runtime-fault channel: `env.fpy_fault`, raised +//! by compiler-emitted guards (division by zero, arithmetic overflow) +//! with a DirectiveErrorCode. Kept separate so a sequence calling +//! exit(10) can never impersonate a DOMAIN_ERROR fault. +//! +//! The process exit status only distinguishes "ran cleanly" (0) from +//! "harness/runtime fault" (2), so error codes are not conflated with traps. //! //! The `env.{pow,fmod,log}` host imports the fpy LLVM backend may emit are //! provided here, backed by libm so they match the C/IEEE semantics the LLVM -//! intrinsics lower to. `env.fpy_exit` is provided to terminate the sequence. +//! intrinsics lower to. use std::alloc::Layout; use std::cell::Cell; @@ -129,12 +133,16 @@ fn wasm_alloc() -> spacewasm::Rc { /// `pow`/`fmod`/`log` are backed by libm so edge cases (e.g. `pow(0, -1)` -> /// +inf, domain errors -> NaN) match what the LLVM intrinsics produce. /// -/// `fpy_exit(code)` ends the whole sequence: it records the code into -/// *exit_code* and unwinds the interpreter with a host trap, so exit() and a -/// failing assert terminate the program from any call depth (a `ret` would only -/// unwind the current function). The recorded code -- not the trap -- carries -/// the result; code 0 is a normal exit, nonzero a fault. -fn fpy_host_module(exit_code: Rc>>) -> HostModule { +/// `fpy_exit(code)` and `fpy_fault(code)` both end the whole sequence: they +/// record the code into their channel's cell and unwind the interpreter with +/// a host trap, so they terminate the program from any call depth (a `ret` +/// would only unwind the current function). The recorded code -- not the +/// trap -- carries the result. The two cells keep the user-exit and +/// runtime-fault channels distinguishable. +fn fpy_host_module( + exit_code: Rc>>, + fault_code: Rc>>, +) -> HostModule { fn arg_f64(args: &[Value], i: usize) -> f64 { match args[i] { Value::F64(v) => v, @@ -169,20 +177,43 @@ fn fpy_host_module(exit_code: Rc>>) -> HostModule { // Unwind the whole interpreter; run() reads the code back. ControlFlow::Break(HostFunctionBreak::Trap) }), + HostFunction::new("fpy_fault", "i".into(), "".into(), move |_, args| { + let code = match args[0] { + Value::I32(v) => v, + other => panic!("expected i32 fault code, got {other:?}"), + }; + fault_code.set(Some(code)); + // Unwind the whole interpreter; run() reads the code back. + ControlFlow::Break(HostFunctionBreak::Trap) + }), ], memory: spacewasm::vec![], table: spacewasm::vec![], } } -fn run(wasm_path: &str, entry: &str) -> Result { +/// A finished sequence's outcome: which channel ended it, and its code. +enum Outcome { + /// User-exit channel: exit(), assert, or fpy_main returning normally. + Exit(i32), + /// Runtime-fault channel: a compiler-emitted guard raised a + /// DirectiveErrorCode (division by zero, arithmetic overflow, ...). + Fault(i32), +} + +fn run(wasm_path: &str, entry: &str) -> Result { let wasm = std::fs::read(wasm_path).map_err(|e| format!("read {wasm_path}: {e}"))?; - // fpy_exit writes the sequence's exit code here and unwinds the interpreter. + // fpy_exit / fpy_fault write the sequence's code into their channel's + // cell and unwind the interpreter. let exit_code: Rc>> = Rc::new(Cell::new(None)); + let fault_code: Rc>> = Rc::new(Cell::new(None)); - let mut store = spacewasm::Store::new(256, [fpy_host_module(exit_code.clone())]) - .map_err(|e| format!("store: {e:?}"))?; + let mut store = spacewasm::Store::new( + 256, + [fpy_host_module(exit_code.clone(), fault_code.clone())], + ) + .map_err(|e| format!("store: {e:?}"))?; let mut code_builder = CodeBuilder::<256>::default(); let mut stream: ByteStream = ByteStream::new(&wasm); @@ -243,19 +274,22 @@ fn run(wasm_path: &str, entry: &str) -> Result { let interpreter = spacewasm::Interpreter::default(); let result = interpreter.run(&text, &mut state, 10_000_000); - // If fpy_exit ran, its recorded code is authoritative -- it unwinds via a - // host trap, so the interpreter result is a trap we must not treat as a - // fault. exit()/assert take this path; falling off the end of fpy_main does - // not (it returns normally below). + // If fpy_exit or fpy_fault ran, the recorded code is authoritative -- they + // unwind via a host trap, so the interpreter result is a trap we must not + // treat as a harness fault. Falling off the end of fpy_main takes neither + // path (it returns normally below). + if let Some(code) = fault_code.get() { + return Ok(Outcome::Fault(code)); + } if let Some(code) = exit_code.get() { - return Ok(code); + return Ok(Outcome::Exit(code)); } match result { InterpreterResult::Instruction(InterpreterBreak::Finished) => { let raw: spacewasm::RawValue = state.result.ok_or("entry returned no value")?; match raw.to_value(spacewasm::ValType::I32) { - Value::I32(code) => Ok(code), + Value::I32(code) => Ok(Outcome::Exit(code)), other => Err(format!("entry returned non-i32: {other:?}")), } } @@ -274,8 +308,12 @@ fn main() -> ExitCode { let entry = args.next().unwrap_or_else(|| "fpy_main".to_string()); match run(&wasm_path, &entry) { - Ok(code) => { - println!("{code}"); + Ok(Outcome::Exit(code)) => { + println!("exit {code}"); + ExitCode::SUCCESS + } + Ok(Outcome::Fault(code)) => { + println!("fault {code}"); ExitCode::SUCCESS } Err(msg) => { diff --git a/verify/arith_properties.py b/verify/arith_properties.py new file mode 100644 index 0000000..bf98474 --- /dev/null +++ b/verify/arith_properties.py @@ -0,0 +1,1173 @@ +#!/usr/bin/env python3 +"""Z3 specification of fpy's arithmetic, comparison, and boolean operators. + +This is the arithmetic counterpart of cast_properties.py. It formalizes the +two-stage semantics the compiler implements for every operator expression: + + 1. TYPE LEVEL (compile time, plain Python over Ty): intermediate_type(op, + arg_types) picks the type both operands are coerced to and in which the + operation computes. It mirrors PickTypesAndResolveFields. + pick_intermediate_type (src/fpy/semantics.py) restricted to runtime + types; check_intermediate_type_matches_compiler verifies the two agree + on every (op, types) combination, so the formal rules cannot drift from + the implementation silently. + + 2. VALUE LEVEL (run time, Z3 terms): each operator is a total function + op(a: Value, b: Value) -> Result. A Value is a Z3 term tagged with its + fpy type. A Result is the value of the expression plus a `halted` + condition: True on exactly the inputs where the program must end with a + runtime error. Operands are first coerced to the intermediate type; the + value map of every coercion the type system admits is the cast function + from cast_properties.py (casting is just explicit coercion, MATH.md). + +Results model *pure* expression evaluation. Sequencing concerns -- whether +the operands themselves halted, and/or short-circuit evaluation of `and`/ +`or` -- compose outside these functions in a future program-level spec. + +The rules encoded (MATH.md, "Arithmetic on I64, U64" / "Arithmetic on F64"): + + * F64 arithmetic is IEEE-754 (RNE) and never halts: x/0.0 is +-inf, + 0.0/0.0 is NaN, overflow rounds to +-inf. + * I64/U64 `+`, `-`, `*`, unary `-` produce the exact mathematical result; + if that result is not representable in the intermediate type the + program halts. Overflow is an error, not a wrap. Unary `-` on an + unsigned operand is a compile error (as in Rust). + * `//` and `%` on integers halt on exactly the same inputs: when the + divisor is 0, and on I64_MIN op -1 (the quotient 2^63 is + unrepresentable; the remainder would be 0 but halts anyway, matching + Rust and keeping the //-% pair coherent). Both are *floored* (Python + semantics): `//` rounds the quotient toward -inf, `%` takes the sign + of the divisor. + * `//` on floats is round-toward-negative of the IEEE quotient. `%` on + floats is C fmod (the exact truncated remainder) followed by one RNE + addition of the divisor when the sign must flip -- exactly what the + LLVM backend emits (frem + fadd) and what CPython computes. x % 0.0 is + NaN, not a halt (CPython raises here; fpy float ops never halt). + * `/` and `**` always compute in F64. `**` is the platform's libm pow, + modeled as an uninterpreted function: deterministic and type-correct, + value otherwise unspecified. + * Comparisons coerce to the intermediate type and yield bool. On floats + they are the IEEE predicates: every comparison involving NaN is False, + except `!=` which is the negation of `==` and hence True. + +Open questions this spec makes visible (marked OQ-n at their encodings): + + OQ-1 Integer overflow halts here, per MATH.md's normative paragraph and + the VM (handle_add/sub/smul return ARITHMETIC_OVERFLOW/UNDERFLOW). + The LLVM backend emits plain add/sub/mul, which WRAP (and one + MATH.md sentence says wrapping). Proving codegen against this spec + will fail until overflow guards exist -- or until the spec flips. + OQ-2 RESOLVED 2026-07-06: unary minus on an unsigned operand is a + compile error (Rust's rule), not a halt-unless-zero. Encoded in + pick_intermediate_type and mirrored in intermediate_type here. + OQ-3 RESOLVED 2026-07-05: float `!=` is Not(==), so NaN != NaN is True + (IEEE, Python, and the VM agree). The LLVM backend used to emit + fcmp `one` (False on NaN); it now emits fcmp une. + OQ-4 RESOLVED 2026-07-05/06: LLVM udiv/sdiv/urem/srem are UB on divisor + 0 (and sdiv/srem on MIN/-1); the backend now guards them: divisor 0 + exits with DOMAIN_ERROR (matching the VM), and MIN // -1 and + MIN % -1 both exit with ARITHMETIC_OVERFLOW (Rust's behavior; the + VM model does the same). + OQ-5 RESOLVED 2026-07-06: float x % 0.0 is NaN -- float ops never halt, + matching IEEE, Rust, and C# (both verified: no trap in any mode). + The VM model's handle_fmod now agrees; CPython raises instead, a + deliberate divergence. The C++ op_fmod still returns DOMAIN_ERROR + (and computes a different formula) -- needs an upstream fix. + +Run: + uv run --with z3-solver python verify/arith_properties.py +""" + +import math +import operator +import random +import sys +from dataclasses import dataclass +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from z3 import ( + RNE, + RTN, + And, + BitVecVal, + BoolSort, + BoolVal, + BVAddNoOverflow, + BVAddNoUnderflow, + BVMulNoOverflow, + BVMulNoUnderflow, + BVSNegNoOverflow, + BVSubNoOverflow, + BVSubNoUnderflow, + Const, + Extract, + FPSort, + FPVal, + Function, + If, + Implies, + Not, + Or, + SignExt, + SRem, + UDiv, + UGE, + UGT, + ULE, + ULT, + URem, + Xor, + ZeroExt, + fpAbs, + fpAdd, + fpDiv, + fpEQ, + fpGEQ, + fpGT, + fpIsNaN, + fpIsNegative, + fpIsZero, + fpLEQ, + fpLT, + fpMul, + fpNaN, + fpNeg, + fpRem, + fpRoundToIntegral, + fpSub, + is_false, + is_true, + simplify, +) + +from cast_properties import ( + F64S, + FAIL, + PASS, + TYPES, + Ty, + cast, + prove, + record, + results, +) + +from fpy.syntax import ( + BOOLEAN_OPERATORS, + COMPARISON_OPS, + NUMERIC_OPERATORS, + BinaryStackOp, + UnaryStackOp, +) + +F64TY = TYPES["F64"] +I64TY = TYPES["I64"] +U64TY = TYPES["U64"] + + +# --- values and results -------------------------------------------------------- + + +@dataclass(frozen=True) +class BoolTy: + """The bool type. Not a Ty: it has no bit-level numeric interpretation + here; bool values are Z3 Booleans.""" + + name: str = "bool" + + @property + def sort(self): + return BoolSort() + + +BOOL = BoolTy() + + +@dataclass(frozen=True) +class Value: + """A runtime value: a Z3 term `expr` of sort ty.sort, tagged with its + fpy type.""" + + ty: object # Ty | BoolTy + expr: object # z3 term of sort ty.sort + + +@dataclass(frozen=True) +class Result: + """The outcome of evaluating one operator on already-evaluated operands: + the value produced, and the condition under which evaluation instead + halts the program with a runtime error. When halted is True the value + is unconstrained (don't-care).""" + + value: Value + halted: object # z3 Bool + + +# --- type level: the intermediate type rules ------------------------------------- + + +def intermediate_type(op, arg_tys): + """The type the operands of `op` are coerced to and in which the op + computes. Returns None where the compiler rejects the program. + + Mirrors PickTypesAndResolveFields.pick_intermediate_type restricted to + the runtime types (the 10 specific numeric types and bool). The + arb-precision Int/Float types only occur in compile-time constant + folding, and non-numeric ==/!= (structs, arrays, enums, time) is a + byte-comparison with no arithmetic content; both are out of scope here. + + The rules: + 1. Boolean operators compute in bool (operand bool-ness is enforced + by coercion, exactly as in the compiler). + 2. bool == bool and bool != bool compute in bool. + 3. Everything else requires numeric operands. + 4. Unary `-` is undefined for unsigned integers (compile error, as in + Rust: the result is negative for every nonzero operand). + 5. `/` and `**` always compute in F64. + 6. If any operand is a float, the op computes in F64. + 7. Signed and unsigned integers never mix: compile error. + 8. Unsigned operands compute in U64; signed operands in I64. + """ + if op in BOOLEAN_OPERATORS: + return BOOL + if op in (BinaryStackOp.EQUAL, BinaryStackOp.NOT_EQUAL): + if all(isinstance(t, BoolTy) for t in arg_tys): + return BOOL + if not all(isinstance(t, Ty) for t in arg_tys): + return None + # arity distinguishes negation from subtraction: ops are str-valued + # enums and NEGATE == SUBTRACT (both "-") + if ( + len(arg_tys) == 1 + and op == UnaryStackOp.NEGATE + and arg_tys[0].kind == "int" + and not arg_tys[0].signed + ): + return None + if op in (BinaryStackOp.DIVIDE, BinaryStackOp.EXPONENT): + return F64TY + if any(t.kind == "float" for t in arg_tys): + return F64TY + if len(arg_tys) == 2 and arg_tys[0].signed != arg_tys[1].signed: + return None + if any(not t.signed for t in arg_tys): + return U64TY + return I64TY + + +def coercible(frm, to) -> bool: + """Whether the compiler admits an implicit conversion frm -> to: only + identity, lossless integer widening (same signedness), float widening, + and int->float. An operand that is not coercible to the intermediate + type is a compile error (this is where e.g. `1 and 2` is rejected, + since boolean operators pick bool regardless of the operand types).""" + if isinstance(frm, BoolTy) or isinstance(to, BoolTy): + return frm == to + if frm == to: + return True + if frm.kind == "int" and to.kind == "float": + return True + return ( + frm.kind == to.kind + and to.bits >= frm.bits + and (frm.kind == "float" or frm.signed == to.signed) + ) + + +def coerce(v: Value, to) -> Value: + """Coerce an operand to the intermediate type. + + The value map of every admitted coercion is the spec cast function + (casting is merely explicit coercion). Note int->F64 genuinely rounds + (RNE) for |x| > 2^53: `/` on two I64s divides *rounded* operands. + """ + assert coercible(v.ty, to), f"cannot coerce {v.ty.name} to {to.name}" + if v.ty == to: + return v + return Value(to, cast(v.expr, v.ty, to)) + + +# --- value level: integer ops ---------------------------------------------------- + + +def int_exact(ty: Ty, extra: int, fn, *args): + """The exact mathematical result of fn over [[ty]] operands, with halt + on unrepresentability (OQ-1). + + Encoding: embed the operands in a ring wide enough that fn cannot wrap + (extra=1 bit for +, -, unary -; extra=ty.bits for *), apply fn there -- + which therefore equals the mathematical operation -- and take the low + ty.bits as the value. The mathematical result is representable in + [[ty]] iff re-extending the value gives back the wide result; halt on + exactly the complement. check_int_overflow_encoding proves this halt + condition equals Z3's independent no-overflow/no-underflow predicates. + """ + ext = SignExt if ty.signed else ZeroExt + wide = fn(*(ext(extra, a) for a in args)) + value = Extract(ty.bits - 1, 0, wide) + halted = ext(extra, value) != wide + return value, halted + + +def int_floor_div(a, b, ty: Ty): + """Floored integer division (Python //). Halts on b == 0, and for + signed types on MIN // -1, whose mathematical quotient 2^(bits-1) is + unrepresentable (OQ-1, OQ-4).""" + n = ty.bits + zero = BitVecVal(0, n) + if not ty.signed: + # quotient of non-negatives is non-negative: truncation == floor + return UDiv(a, b), b == zero + halted = Or( + b == zero, + And(a == BitVecVal(ty.min, n), b == BitVecVal(-1, n)), + ) + q = a / b # SMT bvsdiv: truncates toward zero + r = SRem(a, b) # sign of the dividend + # truncation differs from floor exactly when the exact quotient is + # negative (operand signs differ <=> xor is negative) and inexact + # (nonzero remainder); there truncation overshoots the floor by one + adjust = And(r != zero, (a ^ b) < zero) + return If(adjust, q - 1, q), halted + + +def int_mod(a, b, ty: Ty): + """Floored integer modulo (Python %): the result takes the sign of the + divisor. Halts on b == 0 and, for signed types, on MIN % -1 -- exactly + the same condition as //, and Rust's behavior. (The mathematical + remainder there would be 0, but the operation is UB in C++/LLVM, and + defining `a // b` and `a % b` to halt together keeps the pair coherent: + wherever one is defined, a == (a // b) * b + (a % b).) + + check_int_divmod_props proves this equals SMT-LIB's floored bvsmod and + satisfies the identity with the sign/range conditions. + """ + zero = BitVecVal(0, ty.bits) + if not ty.signed: + return URem(a, b), b == zero + halted = Or( + b == zero, + And(a == BitVecVal(ty.min, ty.bits), b == BitVecVal(-1, ty.bits)), + ) + r = SRem(a, b) # truncated remainder: sign of the dividend + # flip toward the divisor's sign when nonzero and signs differ + flip = And(r != zero, (r ^ b) < zero) + return If(flip, r + b, r), halted + + +# --- value level: float ops ------------------------------------------------------ + +# `**`: the platform libm's pow. Deterministic and total, but its rounding +# is implementation-defined, so the spec leaves it uninterpreted. Anything +# proved about programs holds for every possible pow; nothing about pow's +# values can be proved, by design. +FPOW = Function("fpy_pow", F64S, F64S, F64S) + + +def float_fmod_trunc(a, b): + """C fmod / LLVM frem: the exact truncated remainder a - trunc(a/b)*b + (computed as if with infinite precision; it is always representable). + + Z3 only exposes fpRem, the IEEE `remainder` whose quotient is rounded + to *nearest* instead of truncated. The two differ by exactly + copysign(|b|, a), precisely when fpRem's nonzero result has the + opposite sign of a; that correction addition is exact because the true + sum (the fmod result) is representable, so RNE does not round. Special + cases (NaN, inf a, zero b -> NaN; fmod(a, inf) = a; zero results keep + the sign of a) come along from fpRem unchanged. + """ + r = fpRem(a, b) + mag = fpAbs(b) + correction = If(fpIsNegative(a), fpNeg(mag), mag) + flip = And( + Not(fpIsNaN(r)), + Not(fpIsZero(r)), + Xor(fpIsNegative(r), fpIsNegative(a)), + ) + return If(flip, fpAdd(RNE(), r, correction), r) + + +def float_mod(a, b): + """Floored float modulo (Python %): truncated fmod, then one RNE + addition of the divisor when the sign must flip toward it. + + This is definitionally what the LLVM backend emits (frem + fadd) and + what CPython computes; unlike the fmod correction, this addition CAN + round (e.g. -1e-300 % 1e300: the exact answer 1e300 - 1e-300 is not + representable), so the rounding is part of the spec. Two deliberate + divergences from CPython: x % 0.0 is NaN, not an error, and an exactly + zero result keeps fmod's zero sign rather than copysign(0, b). + """ + m = float_fmod_trunc(a, b) + flip = And( + Not(fpIsNaN(m)), + Not(fpIsZero(m)), + Xor(fpIsNegative(m), fpIsNegative(b)), + ) + return If(flip, fpAdd(RNE(), m, b), m) + + +# --- value level: the operators --------------------------------------------------- + +NEVER = BoolVal(False) + + +def _compare(op, a, b, ty: Ty): + if ty.kind == "float": + if op == BinaryStackOp.EQUAL: + return fpEQ(a, b) + if op == BinaryStackOp.NOT_EQUAL: + return Not(fpEQ(a, b)) # OQ-3: True when either side is NaN + fp_table = { + BinaryStackOp.LESS_THAN: fpLT, + BinaryStackOp.LESS_THAN_OR_EQUAL: fpLEQ, + BinaryStackOp.GREATER_THAN: fpGT, + BinaryStackOp.GREATER_THAN_OR_EQUAL: fpGEQ, + } + return fp_table[op](a, b) + if op == BinaryStackOp.EQUAL: + return a == b + if op == BinaryStackOp.NOT_EQUAL: + return a != b + int_table = ( + { + BinaryStackOp.LESS_THAN: operator.lt, + BinaryStackOp.LESS_THAN_OR_EQUAL: operator.le, + BinaryStackOp.GREATER_THAN: operator.gt, + BinaryStackOp.GREATER_THAN_OR_EQUAL: operator.ge, + } + if ty.signed + else { + BinaryStackOp.LESS_THAN: ULT, + BinaryStackOp.LESS_THAN_OR_EQUAL: ULE, + BinaryStackOp.GREATER_THAN: UGT, + BinaryStackOp.GREATER_THAN_OR_EQUAL: UGE, + } + ) + return int_table[op](a, b) + + +def _numeric_binop(op, a, b, ty: Ty): + if ty.kind == "float": + rm = RNE() + if op == BinaryStackOp.ADD: + return fpAdd(rm, a, b), NEVER + if op == BinaryStackOp.SUBTRACT: + return fpSub(rm, a, b), NEVER + if op == BinaryStackOp.MULTIPLY: + return fpMul(rm, a, b), NEVER + if op == BinaryStackOp.DIVIDE: + return fpDiv(rm, a, b), NEVER + if op == BinaryStackOp.FLOOR_DIVIDE: + # divide, then floor the quotient (round toward negative) + return fpRoundToIntegral(RTN(), fpDiv(rm, a, b)), NEVER + if op == BinaryStackOp.MODULUS: + return float_mod(a, b), NEVER + assert op == BinaryStackOp.EXPONENT, op + return FPOW(a, b), NEVER + if op == BinaryStackOp.ADD: + return int_exact(ty, 1, operator.add, a, b) + if op == BinaryStackOp.SUBTRACT: + return int_exact(ty, 1, operator.sub, a, b) + if op == BinaryStackOp.MULTIPLY: + return int_exact(ty, ty.bits, operator.mul, a, b) + if op == BinaryStackOp.FLOOR_DIVIDE: + return int_floor_div(a, b, ty) + assert op == BinaryStackOp.MODULUS, op # / and ** are float-only + return int_mod(a, b, ty) + + +def apply_binary(op: BinaryStackOp, a: Value, b: Value) -> Result: + """The value-level semantics of `a op b` on already-evaluated operands.""" + ty = intermediate_type(op, [a.ty, b.ty]) + assert ( + ty is not None + ), f"{op} undefined for {a.ty.name}, {b.ty.name} (compile error)" + + if op in BOOLEAN_OPERATORS: + av, bv = coerce(a, BOOL).expr, coerce(b, BOOL).expr + fn = And if op == BinaryStackOp.AND else Or + return Result(Value(BOOL, fn(av, bv)), NEVER) + + if isinstance(ty, BoolTy): + # bool == bool / bool != bool + eq = a.expr == b.expr + val = eq if op == BinaryStackOp.EQUAL else Not(eq) + return Result(Value(BOOL, val), NEVER) + + av, bv = coerce(a, ty).expr, coerce(b, ty).expr + + if op in COMPARISON_OPS: + return Result(Value(BOOL, _compare(op, av, bv, ty)), NEVER) + + assert op in NUMERIC_OPERATORS, op + value, halted = _numeric_binop(op, av, bv, ty) + return Result(Value(ty, value), halted) + + +def apply_unary(op: UnaryStackOp, a: Value) -> Result: + """The value-level semantics of `op a` on an already-evaluated operand.""" + ty = intermediate_type(op, [a.ty]) + assert ty is not None, f"{op} undefined for {a.ty.name} (compile error)" + + if op == UnaryStackOp.NOT: + return Result(Value(BOOL, Not(coerce(a, BOOL).expr)), NEVER) + + av = coerce(a, ty).expr + if op == UnaryStackOp.IDENTITY: + return Result(Value(ty, av), NEVER) + + assert op == UnaryStackOp.NEGATE, op + if ty.kind == "float": + return Result(Value(ty, fpNeg(av)), NEVER) + # unsigned operands were rejected by intermediate_type, so ty is a + # signed type here and the only unrepresentable result is -MIN + assert ty.signed + value, halted = int_exact(ty, 1, operator.neg, av) + return Result(Value(ty, value), halted) + + +def add(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.ADD, a, b) + + +def sub(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.SUBTRACT, a, b) + + +def mul(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.MULTIPLY, a, b) + + +def div(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.DIVIDE, a, b) + + +def floor_div(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.FLOOR_DIVIDE, a, b) + + +def mod(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.MODULUS, a, b) + + +def exponent(a: Value, b: Value) -> Result: + return apply_binary(BinaryStackOp.EXPONENT, a, b) + + +def neg(a: Value) -> Result: + return apply_unary(UnaryStackOp.NEGATE, a) + + +# --- checks ---------------------------------------------------------------------- + +ALL_SPEC_TYS = list(TYPES.values()) + [BOOL] + + +def check_intermediate_type_matches_compiler(): + """The formal type rules agree with the production compiler on every + (op, operand types) combination -- pick_intermediate_type itself is the + oracle, so the spec cannot silently drift from the implementation.""" + from fpy import types as fpy_types + from fpy.semantics import PickTypesAndResolveFields + + compiler = PickTypesAndResolveFields() + + def to_fpy(t): + if t is None: + return None + if isinstance(t, BoolTy): + return fpy_types.BOOL + return getattr(fpy_types, t.name) + + bad = [] + n = 0 + for op in BinaryStackOp: + for t1 in ALL_SPEC_TYS: + for t2 in ALL_SPEC_TYS: + got = compiler.pick_intermediate_type([to_fpy(t1), to_fpy(t2)], op) + want = to_fpy(intermediate_type(op, [t1, t2])) + n += 1 + if got != want: + bad.append( + f"{t1.name} {op.value} {t2.name}: compiler={got} spec={want}" + ) + for op in UnaryStackOp: + for t1 in ALL_SPEC_TYS: + got = compiler.pick_intermediate_type([to_fpy(t1)], op) + want = to_fpy(intermediate_type(op, [t1])) + n += 1 + if got != want: + bad.append(f"{op.value} {t1.name}: compiler={got} spec={want}") + if bad: + record(FAIL, "type rules == pick_intermediate_type", "; ".join(bad[:10])) + else: + record( + PASS, "type rules == pick_intermediate_type", f"all {n} combinations agree" + ) + + +def check_well_formed(): + """Every legal (op, types) combination builds a closed Result: the value + has the result type's sort, halted is Boolean, and no free variable + other than the two operands appears.""" + from z3.z3util import get_vars + + bad = [] + n = 0 + for op in BinaryStackOp: + for t1 in ALL_SPEC_TYS: + for t2 in ALL_SPEC_TYS: + ty = intermediate_type(op, [t1, t2]) + if ty is None or not (coercible(t1, ty) and coercible(t2, ty)): + continue + n += 1 + x = Const(f"wf_a_{t1.name}", t1.sort) + y = Const(f"wf_b_{t2.name}", t2.sort) + res = apply_binary(op, Value(t1, x), Value(t2, y)) + want_sort = ( + BoolSort() + if op in COMPARISON_OPS or op in BOOLEAN_OPERATORS + else ty.sort + ) + if res.value.expr.sort() != want_sort: + bad.append( + f"{t1.name} {op.value} {t2.name}: sort {res.value.expr.sort()}" + ) + continue + if res.halted.sort() != BoolSort(): + bad.append(f"{t1.name} {op.value} {t2.name}: halted sort") + continue + fv = get_vars(res.value.expr) + get_vars(res.halted) + if not all( + v.eq(x) or v.eq(y) or v.decl().name() == "fpy_pow" for v in fv + ): + bad.append(f"{t1.name} {op.value} {t2.name}: free vars {fv}") + for op in UnaryStackOp: + for t1 in ALL_SPEC_TYS: + ty = intermediate_type(op, [t1]) + if ty is None or not coercible(t1, ty): + continue + n += 1 + x = Const(f"wf_u_{t1.name}", t1.sort) + res = apply_unary(op, Value(t1, x)) + want_sort = BoolSort() if op == UnaryStackOp.NOT else ty.sort + if res.value.expr.sort() != want_sort: + bad.append(f"{op.value} {t1.name}: sort {res.value.expr.sort()}") + continue + fv = get_vars(res.value.expr) + get_vars(res.halted) + if not all(v.eq(x) for v in fv): + bad.append(f"{op.value} {t1.name}: free vars {fv}") + if bad: + record(FAIL, "well-formed results", "; ".join(bad[:10])) + else: + record( + PASS, "well-formed results", f"all {n} legal combinations closed and sorted" + ) + + +def check_int_overflow_encoding(): + """The widening halt condition of int_exact equals Z3's independent + built-in no-overflow/no-underflow predicates -- two unrelated encodings + of 'the mathematical result is unrepresentable'. + + The encoding is uniform in the bit width (the same SignExt/Extract + structure at every width), so the multiply equivalence -- nonlinear and + beyond Z3 for signed widths above 16 -- is proved at the widths Z3 can + finish, and 64-bit multiply is additionally cross-checked concretely + against Python in check_int_ops_match_python.""" + for tyn in ("I64", "U64", "I8", "U8"): + ty = TYPES[tyn] + a = Const(f"ovf_a_{tyn}", ty.sort) + b = Const(f"ovf_b_{tyn}", ty.sort) + s = bool(ty.signed) + + _, h_add = int_exact(ty, 1, operator.add, a, b) + ok_add = ( + And(BVAddNoOverflow(a, b, True), BVAddNoUnderflow(a, b)) + if s + else BVAddNoOverflow(a, b, False) + ) + prove(f"halt(+) == Z3 overflow predicates {tyn}", h_add == Not(ok_add)) + + _, h_sub = int_exact(ty, 1, operator.sub, a, b) + ok_sub = ( + And(BVSubNoOverflow(a, b), BVSubNoUnderflow(a, b, True)) + if s + else BVSubNoUnderflow(a, b, False) + ) + prove(f"halt(-) == Z3 overflow predicates {tyn}", h_sub == Not(ok_sub)) + + if s: + # (negation of unsigned operands is a compile error, so the + # encoding is only an operator semantics for signed types) + _, h_neg = int_exact(ty, 1, operator.neg, a) + prove( + f"halt(unary -) == Z3 overflow predicate {tyn}", + h_neg == Not(BVSNegNoOverflow(a)), + ) + + # and when it does not halt, the value is the plain wrapped op + prove( + f"value(+) == bvadd {tyn}", int_exact(ty, 1, operator.add, a, b)[0] == a + b + ) + prove( + f"value(*) == bvmul {tyn}", + int_exact(ty, ty.bits, operator.mul, a, b)[0] == a * b, + ) + + for tyn in ("I8", "U8", "I16", "U16", "U32", "U64"): + ty = TYPES[tyn] + a = Const(f"ovfm_a_{tyn}", ty.sort) + b = Const(f"ovfm_b_{tyn}", ty.sort) + _, h_mul = int_exact(ty, ty.bits, operator.mul, a, b) + ok_mul = ( + And(BVMulNoOverflow(a, b, True), BVMulNoUnderflow(a, b)) + if ty.signed + else BVMulNoOverflow(a, b, False) + ) + prove(f"halt(*) == Z3 overflow predicates {tyn}", h_mul == Not(ok_mul)) + + +def check_int_divmod_props(): + """Floored // and % are each other's complements and % is SMT-LIB's + floored bvsmod: // and % halt on exactly the same inputs, and wherever + they are defined, + a == (a // b) * b + (a % b), + the remainder has the divisor's sign (or is 0), and |a % b| < |b|. + + Nonlinear division proofs are expensive, and the encoding is uniform in + the width, so the div/mod identity is proved at 8 bits and the cheaper + remainder properties through 16 bits (bvsmod also at 64); 64-bit // and + % are cross-checked concretely against Python in + check_int_ops_match_python.""" + a64 = Const("dm_a_I64", I64TY.sort) + b64 = Const("dm_b_I64", I64TY.sort) + r64, _ = int_mod(a64, b64, I64TY) + prove("a%b == bvsmod I64", Implies(b64 != BitVecVal(0, 64), r64 == a64 % b64)) + for tyn in ("I8", "U8", "I16", "U16"): + ty = TYPES[tyn] + a = Const(f"dm_a_{tyn}", ty.sort) + b = Const(f"dm_b_{tyn}", ty.sort) + zero = BitVecVal(0, ty.bits) + q, hq = int_floor_div(a, b, ty) + r, hr = int_mod(a, b, ty) + if ty.bits == 8: + prove(f"a == (a//b)*b + a%b {tyn}", Implies(Not(hq), a == q * b + r)) + if ty.signed: + # z3py maps % on signed bitvectors to SMT-LIB's floored bvsmod + prove(f"a%b == bvsmod {tyn}", Implies(b != zero, r == a % b)) + prove( + f"a%b sign and range {tyn}", + Implies( + b != zero, + And( + Implies(b > zero, And(r >= zero, r < b)), + Implies(b < zero, And(r <= zero, r > b)), + ), + ), + ) + else: + prove(f"a%b range {tyn}", Implies(b != zero, ULT(r, b))) + prove(f"// and % halt together {tyn}", hq == hr) + + +def check_narrow_types_never_halt(): + """Widening to the 64-bit intermediate type protects narrow operands: + +, -, * on (up to) 32-bit operands can never overflow the intermediate + type, hence never halt. (Signed - unsigned mixes are compile errors, and + U64/I64 operands genuinely can halt: see the spot checks.)""" + for t1n, t2n in [("U8", "U32"), ("U32", "U32"), ("I16", "I32"), ("I32", "I32")]: + t1, t2 = TYPES[t1n], TYPES[t2n] + x = Const(f"nh_a_{t1n}_{t2n}", t1.sort) + y = Const(f"nh_b_{t1n}_{t2n}", t2.sort) + a, b = Value(t1, x), Value(t2, y) + for name, fn in [("+", add), ("-", sub), ("*", mul)]: + if name == "-" and not t1.signed: + # unsigned subtraction CAN halt even for narrow types + continue + prove(f"{t1n} {name} {t2n} never halts", Not(fn(a, b).halted)) + # ... and unsigned narrow subtraction halts exactly when a < b + t = TYPES["U8"] + x = Const("nh_sub_a", t.sort) + y = Const("nh_sub_b", t.sort) + res = sub(Value(t, x), Value(t, y)) + prove("U8 - U8 halts iff a < b", res.halted == ULT(x, y)) + + +def check_float_mod_props(): + """Sanity of the fmod construction: for finite a and nonzero finite b, + fmod is zero or has the sign of a, and its magnitude is < |b|. + + The construction is sort-generic and fpRem proofs blow up already at + F32, so this is proved on the 16-bit IEEE format; F64 behavior is + cross-checked concretely against math.fmod in + check_float_ops_match_python.""" + f16s = FPSort(5, 11) + a = Const("fm_a", f16s) + b = Const("fm_b", f16s) + m = float_fmod_trunc(a, b) + inputs_ok = And( + Not(fpIsNaN(a)), + Not(fpIsNaN(b)), + fpLT(fpAbs(a), FPVal(float("inf"), f16s)), + fpLT(fpAbs(b), FPVal(float("inf"), f16s)), + Not(fpIsZero(b)), + ) + prove( + "fmod sign == sign(a) or zero (F16)", + Implies(inputs_ok, Or(fpIsZero(m), Not(Xor(fpIsNegative(m), fpIsNegative(a))))), + ) + prove("fmod magnitude < |b| (F16)", Implies(inputs_ok, fpLT(fpAbs(m), fpAbs(b)))) + + +# --- concrete cross-checks vs Python ---------------------------------------------- + + +def iv(x: int, tyn: str) -> Value: + ty = TYPES[tyn] + return Value(ty, BitVecVal(x, ty.bits)) + + +def fv(x: float, tyn: str = "F64") -> Value: + ty = TYPES[tyn] + return Value(ty, FPVal(x, ty.sort)) + + +def concrete_int(expr, ty: Ty) -> int: + v = simplify(expr) + return v.as_signed_long() if ty.signed else v.as_long() + + +def concrete_bool(expr) -> bool: + v = simplify(expr) + assert is_true(v) or is_false(v), v + return is_true(v) + + +def fp_matches(term, expected: float) -> bool: + """Concrete FP term equals the Python float (fpEQ, so +-0 agree; NaN + matches NaN).""" + if math.isnan(expected): + return concrete_bool(fpIsNaN(term)) + return concrete_bool(fpEQ(term, FPVal(expected, term.sort()))) + + +def check_int_ops_match_python(): + """The integer operators agree with Python's unbounded-int arithmetic: + same value when the mathematical result is representable, halt exactly + when it is not (//, % additionally halt per their divisor conditions).""" + random.seed(0) + py_ops = { + "+": operator.add, + "-": operator.sub, + "*": operator.mul, + "//": operator.floordiv, + "%": operator.mod, + } + spec_ops = {"+": add, "-": sub, "*": mul, "//": floor_div, "%": mod} + for tyn in ("I64", "U64"): + ty = TYPES[tyn] + corners = {ty.min, ty.min + 1, 0, 1, 2, ty.max - 1, ty.max} + if ty.signed: + corners |= {-1, -2} + samples = sorted(corners) + [ + random.randrange(ty.min, ty.max + 1) for _ in range(28) + ] + pairs = [(x, y) for x in samples for y in samples[:12]] + for op_name, py_fn in py_ops.items(): + bad = None + for x, y in pairs: + res = spec_ops[op_name](iv(x, tyn), iv(y, tyn)) + halted = concrete_bool(res.halted) + if op_name in ("//", "%"): + # // and % halt together: zero divisor, or MIN op -1 + want_halt = y == 0 or (ty.signed and x == ty.min and y == -1) + else: + exact = py_fn(x, y) + want_halt = not (ty.min <= exact <= ty.max) + if halted != want_halt: + bad = f"{x} {op_name} {y}: halted={halted}, want {want_halt}" + break + if not want_halt: + got = concrete_int(res.value.expr, ty) + want = py_fn(x, y) + if got != want: + bad = f"{x} {op_name} {y}: {got}, python says {want}" + break + if bad: + record(FAIL, f"{tyn} {op_name} == python", bad) + else: + record( + PASS, f"{tyn} {op_name} == python", f"{len(pairs)} sampled pairs" + ) + + +def check_float_ops_match_python(): + """The float operators agree with CPython's float arithmetic (which is + the host's IEEE-754 double) on sampled values, including // and the + fmod-based %.""" + random.seed(1) + interesting = [ + 0.0, + -0.0, + 1.0, + -1.0, + 2.0, + 0.5, + -0.5, + 7.5, + -7.5, + 1e300, + -1e300, + 1e-300, + -1e-300, + 2.0**53, + -(2.0**53), + float("inf"), + -float("inf"), + float("nan"), + 3.141592653589793, + -2.718281828459045, + ] + samples = interesting + [random.uniform(-1e6, 1e6) for _ in range(12)] + + def py_floor_div(x, y): + # the spec's formula computed in host IEEE arithmetic: ONE division, + # then floor. (CPython's own float // is fmod-based and can differ + # in the last ulp, so it is not the reference here.) + try: + q = x / y + except ZeroDivisionError: + if x == 0.0 or math.isnan(x): + return float("nan") + return math.copysign(float("inf"), x) * math.copysign(1.0, y) + if math.isnan(q) or math.isinf(q): + return q + return float(math.floor(q)) + + def py_mod(x, y): + # CPython float_rem: fmod, then add the divisor back on sign + # mismatch (we keep fmod's zero sign where CPython copysigns it to + # y, but fpEQ treats +-0 as equal so the comparison still holds) + if y == 0.0 or math.isnan(x) or math.isnan(y) or math.isinf(x): + return float("nan") + m = math.fmod(x, y) # fmod(finite, +-inf) == x, matching the spec + if m != 0.0 and (m < 0.0) != (y < 0.0): + m = m + y + return m + + checks = [ + ("+", add, operator.add), + ("-", sub, operator.sub), + ("*", mul, operator.mul), + ("/", div, None), + ("//", floor_div, py_floor_div), + ("%", mod, py_mod), + ] + for op_name, spec_fn, py_fn in checks: + bad = None + for x in samples: + for y in samples[:14]: + if py_fn is None: # division: python raises on /0 + if y == 0.0: + want = ( + float("nan") + if (x == 0.0 or math.isnan(x)) + else math.copysign(float("inf"), x) * math.copysign(1.0, y) + ) + else: + want = x / y + else: + try: + want = py_fn(x, y) + except (ZeroDivisionError, ValueError): + want = float("nan") + res = spec_fn(fv(x), fv(y)) + assert concrete_bool(Not(res.halted)) + if not fp_matches(res.value.expr, want): + bad = f"{x!r} {op_name} {y!r}: want {want!r}, got {simplify(res.value.expr)}" + break + if bad: + break + if bad: + record(FAIL, f"F64 {op_name} == python", bad) + else: + record( + PASS, f"F64 {op_name} == python", f"{len(samples) * 14} sampled pairs" + ) + + # fmod itself against math.fmod + bad = None + for x in samples: + for y in samples[:14]: + try: + want = math.fmod(x, y) + except ValueError: + want = float("nan") + if not fp_matches(float_fmod_trunc(fv(x).expr, fv(y).expr), want): + bad = f"fmod({x!r}, {y!r}) != {want!r}" + break + if bad: + break + record( + FAIL if bad else PASS, "float_fmod_trunc == math.fmod", bad or "sampled pairs" + ) + + +def check_spot_semantics(): + """Hand-picked cases that pin down the deliberate design decisions.""" + spots_int = [ + # floored division and modulo (Python, not C, semantics) + (floor_div(iv(-7, "I64"), iv(2, "I64")), -4, False, "-7 // 2 == -4"), + (floor_div(iv(7, "I64"), iv(-2, "I64")), -4, False, "7 // -2 == -4"), + (mod(iv(-7, "I64"), iv(2, "I64")), 1, False, "-7 % 2 == 1"), + (mod(iv(7, "I64"), iv(-2, "I64")), -1, False, "7 % -2 == -1"), + # OQ-1: overflow is a halt, not a wrap + ( + add(iv(TYPES["I64"].max, "I64"), iv(1, "I64")), + None, + True, + "I64_MAX + 1 halts", + ), + (sub(iv(0, "U64"), iv(1, "U64")), None, True, "U64 0 - 1 halts"), + ( + mul(iv(1 << 32, "U64"), iv(1 << 32, "U64")), + None, + True, + "U64 2^32 * 2^32 halts", + ), + ( + floor_div(iv(TYPES["I64"].min, "I64"), iv(-1, "I64")), + None, + True, + "I64_MIN // -1 halts", + ), + ( + mod(iv(TYPES["I64"].min, "I64"), iv(-1, "I64")), + None, + True, + "I64_MIN % -1 halts (like //, like Rust)", + ), + (floor_div(iv(1, "I64"), iv(0, "I64")), None, True, "1 // 0 halts"), + (mod(iv(1, "U64"), iv(0, "U64")), None, True, "1 % 0 halts"), + (neg(iv(TYPES["I64"].min, "I64")), None, True, "-I64_MIN halts"), + # widening: narrow ops that would wrap in their own width don't halt + (add(iv(200, "U8"), iv(100, "U8")), 300, False, "U8 200 + 100 == 300 (in U64)"), + ( + mul(iv(-(2**31), "I32"), iv(2, "I32")), + -(2**32), + False, + "I32 min * 2 ok in I64", + ), + ] + for res, want_val, want_halt, label in spots_int: + halted = concrete_bool(res.halted) + ok = halted == want_halt and ( + want_halt or concrete_int(res.value.expr, res.value.ty) == want_val + ) + record( + PASS if ok else FAIL, + f"spot {label}", + "ok" if ok else f"halted={halted}, value={simplify(res.value.expr)}", + ) + + # negation of an unsigned operand is rejected at the type level (as in + # Rust), not turned into a runtime halt + try: + neg(iv(5, "U32")) + record(FAIL, "spot -U32(5) is a compile error", "spec accepted it") + except AssertionError: + record(PASS, "spot -U32(5) is a compile error", "rejected at type level") + + spots_float = [ + # / never halts; IEEE specials + ( + div(iv(1, "I64"), iv(0, "I64")), + float("inf"), + "1 / 0 == +inf (computed in F64)", + ), + (div(fv(0.0), fv(0.0)), float("nan"), "0.0 / 0.0 == NaN"), + (div(fv(-1.0), fv(0.0)), -float("inf"), "-1.0 / 0.0 == -inf"), + # float floor div floors toward -inf + (floor_div(fv(-7.0), fv(2.0)), -4.0, "-7.0 // 2.0 == -4.0"), + (floor_div(fv(7.5), fv(2.0)), 3.0, "7.5 // 2.0 == 3.0"), + # floored float modulo + (mod(fv(7.5), fv(2.0)), 1.5, "7.5 % 2.0 == 1.5"), + (mod(fv(-7.5), fv(2.0)), 0.5, "-7.5 % 2.0 == 0.5"), + (mod(fv(1.0), fv(0.0)), float("nan"), "1.0 % 0.0 == NaN (no halt)"), + # int operands of / are rounded to F64 first (coercion rounds) + ( + div(iv((1 << 62) + 1, "I64"), iv(1, "I64")), + float((1 << 62) + 1), + "big I64 / 1 rounds", + ), + ] + for res, want, label in spots_float: + ok = concrete_bool(Not(res.halted)) and fp_matches(res.value.expr, want) + record( + PASS if ok else FAIL, + f"spot {label}", + "ok" if ok else f"got {simplify(res.value.expr)}", + ) + + nan = Value(F64TY, fpNaN(F64S)) + spots_bool = [ + (apply_binary(BinaryStackOp.EQUAL, nan, nan), False, "NaN == NaN is False"), + ( + apply_binary(BinaryStackOp.NOT_EQUAL, nan, nan), + True, + "NaN != NaN is True (OQ-3)", + ), + ( + apply_binary(BinaryStackOp.LESS_THAN, nan, fv(1.0)), + False, + "NaN < 1.0 is False", + ), + ( + apply_binary(BinaryStackOp.GREATER_THAN_OR_EQUAL, nan, fv(1.0)), + False, + "NaN >= 1.0 is False", + ), + # mixed-width comparison happens at the common widened type + ( + apply_binary(BinaryStackOp.LESS_THAN, iv(-1, "I8"), iv(1, "I64")), + True, + "I8 -1 < I64 1", + ), + ( + apply_binary(BinaryStackOp.EQUAL, iv(255, "U8"), iv(255, "U64")), + True, + "U8 255 == U64 255", + ), + ] + for res, want, label in spots_bool: + ok = concrete_bool(res.value.expr) == want + record( + PASS if ok else FAIL, f"spot {label}", "ok" if ok else "wrong truth value" + ) + + +def main(): + print("fpy arithmetic operator specification checks (see MATH.md)\n") + check_intermediate_type_matches_compiler() + check_well_formed() + check_int_overflow_encoding() + check_int_divmod_props() + check_narrow_types_never_halt() + check_float_mod_props() + check_int_ops_match_python() + check_float_ops_match_python() + check_spot_semantics() + + n_pass = sum(1 for s, _, _ in results if s == PASS) + n_fail = sum(1 for s, _, _ in results if s == FAIL) + n_warn = sum(1 for s, _, _ in results if s not in (PASS, FAIL)) + print(f"\n{n_pass} passed, {n_fail} failed, {n_warn} unknown/timeout") + sys.exit(1 if n_fail else 0) + + +if __name__ == "__main__": + main() diff --git a/verify/cast_properties.py b/verify/cast_properties.py new file mode 100644 index 0000000..7481f64 --- /dev/null +++ b/verify/cast_properties.py @@ -0,0 +1,668 @@ +#!/usr/bin/env python3 +"""Mechanized consistency checks for the fpy cast specification. + +Spec (MATH_CASTS_DRAFT.md): a cast is a total function +cast_{S->T} : [[S]] -> [[T]] defined by four equations: + + int -> int wrap_T(x) + int -> float round_T(x) (single direct RNE rounding) + float -> float round_T([[x]]) (sign of zero preserved) + float -> int clamp_T(trunc([[x]])) (saturating; NaN -> 0) + +Each equation is encoded as a quantifier-free Z3 term over the SMT bitvector +and IEEE-754 floating-point theories. Totality and determinism (theorem T1) +hold *by construction*: every Z3 expression tree denotes exactly one value of +the result sort for every input, and the SMT FP sorts have a single NaN, +matching the spec's canonical-NaN policy. The checks below verify the +remaining theorems (T2-T7 in the draft). + +GOTCHA encoded here deliberately: SMT-LIB's fp.to_sbv/fp.to_ubv are +*underspecified* for NaN and out-of-range inputs, so the spec functions guard +them and only apply them on their defined domain. The "definedness" checks +prove those guards are sufficient. + +Run: + uv run --with z3-solver python verify/cast_properties.py +""" + +import sys +import time +from dataclasses import dataclass + +from z3 import ( + RNE, + RTZ, + And, + BitVecSort, + BitVecVal, + Const, + Extract, + FPSort, + FPVal, + If, + Implies, + Not, + SignExt, + Solver, + ULE, + ZeroExt, + substitute, + fpGEQ, + fpIsNaN, + fpLEQ, + fpLT, + fpMinusZero, + fpNaN, + fpPlusInfinity, + fpMinusInfinity, + fpPlusZero, + fpRoundToIntegral, + fpSignedToFP, + fpToFP, + fpToSBV, + fpToUBV, + fpUnsignedToFP, + sat, + simplify, + unsat, +) + +F32S = FPSort(8, 24) +F64S = FPSort(11, 53) + + +@dataclass(frozen=True) +class Ty: + name: str + kind: str # 'int' | 'float' + bits: int + signed: bool | None # None for floats + + @property + def sort(self): + if self.kind == "int": + return BitVecSort(self.bits) + return F32S if self.bits == 32 else F64S + + @property + def min(self) -> int: + assert self.kind == "int" + return -(1 << (self.bits - 1)) if self.signed else 0 + + @property + def max(self) -> int: + assert self.kind == "int" + return (1 << (self.bits - 1)) - 1 if self.signed else (1 << self.bits) - 1 + + +TYPES = { + t.name: t + for t in [ + Ty("U8", "int", 8, False), + Ty("U16", "int", 16, False), + Ty("U32", "int", 32, False), + Ty("U64", "int", 64, False), + Ty("I8", "int", 8, True), + Ty("I16", "int", 16, True), + Ty("I32", "int", 32, True), + Ty("I64", "int", 64, True), + Ty("F32", "float", 32, None), + Ty("F64", "float", 64, None), + ] +} +INT_TYPES = [t for t in TYPES.values() if t.kind == "int"] +FLOAT_TYPES = [t for t in TYPES.values() if t.kind == "float"] + + +# --- the four spec equations ------------------------------------------------- + + +def wrap_int_to_int(x, frm: Ty, to: Ty): + """cast(x, int S, int T) = wrap_T(x). + + Structural encoding: extract low bits (narrowing) / extend by the + *source* signedness (widening). check_wrap_matches_mod proves this equals + the mathematical definition r = x mod 2^n. + """ + if to.bits == frm.bits: + return x + if to.bits < frm.bits: + return Extract(to.bits - 1, 0, x) + ext = SignExt if frm.signed else ZeroExt + return ext(to.bits - frm.bits, x) + + +def int_to_float(x, frm: Ty, to: Ty): + """cast(x, int S, float T) = round_T(x): one direct RNE rounding.""" + conv = fpSignedToFP if frm.signed else fpUnsignedToFP + return conv(RNE(), x, to.sort) + + +def float_to_float(x, frm: Ty, to: Ty): + """cast(x, float S, float T) = round_T([[x]]), sign of zero preserved. + + SMT fp.to_fp on FP input is IEEE convertFormat: it preserves the sign of + zero and of underflowed results, and maps NaN to the sort's single NaN -- + exactly the spec including the sign-of-zero clause (checked below). + """ + if frm.name == to.name: + return x + return fpToFP(RNE(), x, to.sort) + + +def float_to_int(x, frm: Ty, to: Ty): + """cast(x, float S, int T) = clamp_T(trunc([[x]])). + + fp.to_sbv/fp.to_ubv are only applied when the RTZ-rounded value provably + fits in T (see check_tosbv_guards_sufficient); all other inputs are + resolved by the clamp arms. The float bounds used in the guards + (+-2^(n-1), 2^n) are powers of two, exactly representable in both F32 and + F64 for n <= 64. + """ + n = to.bits + zero = BitVecVal(0, n) + if to.signed: + lo_f = FPVal(-(2.0 ** (n - 1)), frm.sort) + hi_f = FPVal(2.0 ** (n - 1), frm.sort) + return If( + fpIsNaN(x), + zero, + If( + fpLT(x, lo_f), + BitVecVal(to.min, n), + If( + fpGEQ(x, hi_f), + BitVecVal(to.max, n), + fpToSBV(RTZ(), x, BitVecSort(n)), + ), + ), + ) + hi_f = FPVal(2.0**n, frm.sort) + return If( + fpIsNaN(x), + zero, + If( + fpLT(x, FPVal(0.0, frm.sort)), + zero, # trunc of anything in (-inf, -0] clamps/truncates to 0 + If(fpGEQ(x, hi_f), BitVecVal(to.max, n), fpToUBV(RTZ(), x, BitVecSort(n))), + ), + ) + + +def cast(x, frm: Ty, to: Ty): + if frm.kind == "int" and to.kind == "int": + return wrap_int_to_int(x, frm, to) + if frm.kind == "int" and to.kind == "float": + return int_to_float(x, frm, to) + if frm.kind == "float" and to.kind == "float": + return float_to_float(x, frm, to) + return float_to_int(x, frm, to) + + +# --- check runner ------------------------------------------------------------ + +PASS, FAIL, WARN = "PASS", "FAIL", "WARN" +results: list[tuple[str, str, str]] = [] + + +def record(status, name, detail): + results.append((status, name, detail)) + print(f"[{status}] {name}: {detail}", flush=True) + + +def prove(name, claim, timeout_ms=120_000): + """Expect `claim` valid (negation unsat).""" + s = Solver() + s.set("timeout", timeout_ms) + s.add(Not(claim)) + t0 = time.time() + r = s.check() + dt = time.time() - t0 + if r == unsat: + record(PASS, name, f"proved ({dt:.1f}s)") + elif r == sat: + record(FAIL, name, f"COUNTEREXAMPLE: {s.model()}") + else: + record(WARN, name, f"unknown/timeout after {dt:.1f}s") + + +def find(name, condition, witness_vars, timeout_ms=120_000): + """Expect `condition` satisfiable (a witness exists).""" + s = Solver() + s.set("timeout", timeout_ms) + s.add(condition) + t0 = time.time() + r = s.check() + dt = time.time() - t0 + if r == sat: + m = s.model() + vals = ", ".join( + f"{v} = {m.eval(v, model_completion=True)}" for v in witness_vars + ) + record(PASS, name, f"witness found ({dt:.1f}s): {vals}") + elif r == unsat: + record(FAIL, name, "expected a witness but none exists") + else: + record(WARN, name, f"unknown/timeout after {dt:.1f}s") + + +# --- T1: totality / determinism ---------------------------------------------- + + +def check_totality_note(): + record( + PASS, + "T1 totality+determinism (all 100 pairs)", + "per-model: by construction (quantifier-free terms of total ops; SMT FP " + "sorts have one NaN = canonical-NaN policy). Across models: mechanized " + "below (well-formedness, guard sufficiency, model-independence)", + ) + + +def check_well_formed(): + """T1 mechanized, part 1: every pair constructs a closed function of x. + + For all 100 (S, T) pairs, cast(x, S, T) must (a) build without error -- + the case table is exhaustive, (b) have sort [[T]], and (c) mention no + free variable other than x. (c) rules out the builder accidentally + capturing a constant from another scope, which would make the "function" + depend on a hidden second input. + """ + from z3.z3util import get_vars + + bad = [] + for frm in TYPES.values(): + for to in TYPES.values(): + x = Const(f"wf_{frm.name}_{to.name}", frm.sort) + term = cast(x, frm, to) + if term.sort() != to.sort: + bad.append(f"{frm.name}->{to.name}: sort {term.sort()}") + continue + fv = get_vars(term) + if not (len(fv) == 1 and fv[0].eq(x)): + bad.append(f"{frm.name}->{to.name}: free vars {fv}") + if bad: + record(FAIL, "T1 well-formed terms (all 100 pairs)", "; ".join(bad)) + else: + record( + PASS, + "T1 well-formed terms (all 100 pairs)", + "each term built, sorted [[T]], free vars exactly {x}", + ) + + +def check_model_independence(): + """T1 mechanized, part 2: the spec is ONE function, not one per Z3 model. + + fp.to_sbv/fp.to_ubv are total in every model but SMT-LIB only pins their + value where the RTZ-rounded input is representable; elsewhere each model + may answer differently. Determinism of the spec therefore means: the value + of the cast term never depends on that unpinned region. + + Proof, direct form: take the actual float->int term and substitute the + fp.to_sbv(x) occurrence with two distinct fresh constants g1, g2 standing + for the answers of two arbitrary models, constrained to agree with + fp.to_sbv only on the pinned (representable) region. If the two resulting + terms are provably equal, every model computes the same function. + """ + for f in FLOAT_TYPES: + for t in INT_TYPES: + x = Const(f"mi_{f.name}_{t.name}", f.sort) + n = t.bits + term = cast(x, f, t) + conv = fpToSBV if t.signed else fpToUBV + app = conv(RTZ(), x, BitVecSort(n)) + g1 = Const(f"mi_g1_{f.name}_{t.name}", BitVecSort(n)) + g2 = Const(f"mi_g2_{f.name}_{t.name}", BitVecSort(n)) + t1 = substitute(term, (app, g1)) + t2 = substitute(term, (app, g2)) + assert not t1.eq(t2), "substitution did not fire; wrong app term?" + + # SMT-LIB pins fp.to_sbv/to_ubv exactly when the value obtained by + # rounding toward zero is representable in the target type. + rounded = fpRoundToIntegral(RTZ(), x) + if t.signed: + lo_f = FPVal(-(2.0 ** (n - 1)), f.sort) + hi_f = FPVal(2.0 ** (n - 1), f.sort) + pinned_region = And( + Not(fpIsNaN(x)), fpGEQ(rounded, lo_f), fpLT(rounded, hi_f) + ) + else: + pinned_region = And( + Not(fpIsNaN(x)), + fpGEQ(rounded, FPVal(0.0, f.sort)), + fpLT(rounded, FPVal(2.0**n, f.sort)), + ) + pinned = Implies(pinned_region, And(g1 == app, g2 == app)) + prove( + f"T1 model-independence {f.name}->{t.name}", Implies(pinned, t1 == t2) + ) + + +def check_tosbv_guards_sufficient(): + """The in-range branch never feeds fp.to_sbv/to_ubv an undefined input: + if the guards pass, the RTZ-rounded value provably lies in T's range.""" + for f in FLOAT_TYPES: + for t in INT_TYPES: + x = Const(f"x_{f.name}_{t.name}", f.sort) + n = t.bits + rounded = fpRoundToIntegral(RTZ(), x) + if t.signed: + lo_f = FPVal(-(2.0 ** (n - 1)), f.sort) + hi_f = FPVal(2.0 ** (n - 1), f.sort) + guards = And(Not(fpIsNaN(x)), Not(fpLT(x, lo_f)), Not(fpGEQ(x, hi_f))) + in_range = And(fpGEQ(rounded, lo_f), fpLT(rounded, hi_f)) + else: + hi_f = FPVal(2.0**n, f.sort) + guards = And( + Not(fpIsNaN(x)), + Not(fpLT(x, FPVal(0.0, f.sort))), + Not(fpGEQ(x, hi_f)), + ) + in_range = And(fpGEQ(rounded, FPVal(0.0, f.sort)), fpLT(rounded, hi_f)) + prove( + f"T1 to_sbv/to_ubv guard sufficient {f.name}->{t.name}", + Implies(guards, in_range), + ) + + +# --- T2: identity is by construction (cast returns x for S == T) -------------- + + +def check_identity_note(): + record( + PASS, + "T2 identity casts (all 10 types)", + "by construction: cast(x, T, T) is literally x", + ) + + +# --- T4: saturation boundaries ----------------------------------------------- + + +def check_saturation_boundaries(): + for f in FLOAT_TYPES: + for t in INT_TYPES: + claims = [ + cast(fpNaN(f.sort), f, t) == BitVecVal(0, t.bits), + cast(fpPlusInfinity(f.sort), f, t) == BitVecVal(t.max, t.bits), + cast(fpMinusInfinity(f.sort), f, t) == BitVecVal(t.min, t.bits), + cast(fpPlusZero(f.sort), f, t) == BitVecVal(0, t.bits), + cast(fpMinusZero(f.sort), f, t) == BitVecVal(0, t.bits), + ] + prove(f"T4 NaN/inf/zero boundaries {f.name}->{t.name}", And(*claims)) + + f64, f32 = TYPES["F64"], TYPES["F32"] + spot = [ + # the LLVM-vs-bytecode divergence example from MATH_TODO.txt: + ( + cast(FPVal(300.0, F64S), f64, TYPES["I8"]) == BitVecVal(127, 8), + "I8(300.0) == 127", + ), + ( + cast(FPVal(-300.0, F64S), f64, TYPES["I8"]) == BitVecVal(-128, 8), + "I8(-300.0) == -128", + ), + ( + cast(FPVal(300.0, F64S), f64, TYPES["U8"]) == BitVecVal(255, 8), + "U8(300.0) == 255", + ), + (cast(FPVal(-3.0, F64S), f64, TYPES["U8"]) == BitVecVal(0, 8), "U8(-3.0) == 0"), + (cast(FPVal(-0.5, F64S), f64, TYPES["U8"]) == BitVecVal(0, 8), "U8(-0.5) == 0"), + (cast(FPVal(-1.0, F64S), f64, TYPES["U8"]) == BitVecVal(0, 8), "U8(-1.0) == 0"), + ( + cast(FPVal(126.7, F64S), f64, TYPES["I8"]) == BitVecVal(126, 8), + "I8(126.7) == 126 (trunc)", + ), + ( + cast(FPVal(-126.7, F64S), f64, TYPES["I8"]) == BitVecVal(-126, 8), + "I8(-126.7) == -126 (trunc toward 0)", + ), + # 2^63 is exactly representable in F64 but out of I64 range -> saturate: + ( + cast(FPVal(2.0**63, F64S), f64, TYPES["I64"]) + == BitVecVal((1 << 63) - 1, 64), + "I64(2^63) == I64_MAX", + ), + ( + cast(FPVal(2.0**63, F32S), f32, TYPES["I64"]) + == BitVecVal((1 << 63) - 1, 64), + "I64(F32 2^63) == I64_MAX", + ), + ( + cast(FPVal(1e300, F64S), f64, TYPES["U64"]) == BitVecVal((1 << 64) - 1, 64), + "U64(1e300) == U64_MAX", + ), + ] + for claim, label in spot: + prove(f"T4 spot {label}", claim) + + +# --- T5: monotonicity ---------------------------------------------------------- + + +def check_monotonicity(): + mono_pairs = [ + ("F32", "I8"), + ("F32", "U8"), + ("F32", "I32"), + ("F32", "U32"), + ("F64", "I16"), + ("F64", "I64"), + ("F64", "U64"), + ] + for fn, tn in mono_pairs: + f, t = TYPES[fn], TYPES[tn] + x = Const(f"mx_{fn}_{tn}", f.sort) + y = Const(f"my_{fn}_{tn}", f.sort) + le = (lambda a, b: a <= b) if t.signed else ULE + claim = Implies( + And(Not(fpIsNaN(x)), Not(fpIsNaN(y)), fpLEQ(x, y)), + le(cast(x, f, t), cast(y, f, t)), + ) + prove(f"T5 monotone float->int {fn}->{tn}", claim) + + for fn, tn in [("I64", "F32"), ("U64", "F32"), ("I32", "F64")]: + f, t = TYPES[fn], TYPES[tn] + x = Const(f"nx_{fn}_{tn}", f.sort) + y = Const(f"ny_{fn}_{tn}", f.sort) + hyp = (x <= y) if f.signed else ULE(x, y) + claim = Implies(hyp, fpLEQ(cast(x, f, t), cast(y, f, t))) + prove(f"T5 monotone int->float {fn}->{tn}", claim) + + +# --- T3/T6: exactness and round trips ------------------------------------------ + + +def check_round_trips(): + # S -> F -> S must be the identity when S's values all fit in F's mantissa + for sn, fn in [ + ("I8", "F32"), + ("I16", "F32"), + ("U16", "F32"), + ("I32", "F64"), + ("U32", "F64"), + ]: + s, f = TYPES[sn], TYPES[fn] + v = Const(f"rt_{sn}_{fn}", s.sort) + prove(f"T6 round trip {sn}->{fn}->{sn} == id", cast(cast(v, s, f), f, s) == v) + + # ... and must NOT be the identity when it doesn't fit + for sn, fn in [("I32", "F32"), ("I64", "F64"), ("U64", "F64")]: + s, f = TYPES[sn], TYPES[fn] + v = Const(f"nrt_{sn}_{fn}", s.sort) + find( + f"T6 round trip {sn}->{fn}->{sn} != id (expected counterexample)", + cast(cast(v, s, f), f, s) != v, + [v], + ) + + # float widening is exact: F32 -> F64 -> F32 == id for ALL values, + # including NaN and both zeros (SMT `=` distinguishes +0 from -0, + # so this also proves the sign-of-zero clause for widening) + f32, f64 = TYPES["F32"], TYPES["F64"] + x = Const("wide_x", F32S) + prove( + "T3 F32->F64->F32 == id (incl NaN, +-0)", cast(cast(x, f32, f64), f64, f32) == x + ) + + +def check_sign_of_zero_narrowing(): + f64, f32 = TYPES["F64"], TYPES["F32"] + claims = [ + ( + cast(fpMinusZero(F64S), f64, f32) == fpMinusZero(F32S), + "F32(-0.0 F64) == -0.0", + ), + (cast(fpPlusZero(F64S), f64, f32) == fpPlusZero(F32S), "F32(+0.0 F64) == +0.0"), + # narrowing underflow keeps the sign: -1e-300 is far below F32's + # smallest subnormal, RNE rounds it to zero -- must be MINUS zero + ( + cast(FPVal(-1e-300, F64S), f64, f32) == fpMinusZero(F32S), + "F32(-1e-300) == -0.0", + ), + # narrowing overflow goes to infinity, NOT max-finite (resolves the + # question mark in the old MATH.md draft) + ( + cast(FPVal(1e300, F64S), f64, f32) == fpPlusInfinity(F32S), + "F32(1e300) == +inf", + ), + ( + cast(FPVal(-1e300, F64S), f64, f32) == fpMinusInfinity(F32S), + "F32(-1e300) == -inf", + ), + ] + for claim, label in claims: + prove(f"T4/T3 narrowing {label}", claim) + + +# --- wrap == mod definition ----------------------------------------------------- + + +WRAP_PAIRS = [ + ("I8", "U8"), + ("U8", "I8"), + ("I64", "U8"), + ("I8", "I64"), + ("U8", "I64"), + ("I32", "I64"), + ("U32", "U16"), + ("I64", "U64"), + ("U64", "I8"), + ("I16", "U32"), +] + + +def check_wrap_matches_mod(): + """The structural ext/extract encoding of int->int equals the spec's + mathematical definition wrap_T(z) = z mod 2^n. + + NOTE: the direct encoding Int2BV(BV2Int(x, signed), n) makes Z3 diverge + for every signed-source widening pair (even I8->I64), so the mod-2^n + semantics is checked without the integer theory, two ways: + (a) universally, against the value embedded in a 128-bit ring, where + taking mod 2^n is Extract of the low n bits; + (b) concretely, evaluating the actual Z3 term against plain Python + integer arithmetic -- exhaustive for 8-bit sources, corners plus + random samples for wider sources. + """ + for fn, tn in WRAP_PAIRS: + f, t = TYPES[fn], TYPES[tn] + x = Const(f"w_{fn}_{tn}", f.sort) + ext = SignExt if f.signed else ZeroExt + embedded = ext(128 - f.bits, x) # two's complement value of x, mod 2^128 + mathematical = Extract(t.bits - 1, 0, embedded) # ... mod 2^n + prove( + f"wrap == mod 2^n in BV ring {fn}->{tn}", + wrap_int_to_int(x, f, t) == mathematical, + ) + + +def py_wrap(z: int, to: Ty) -> int: + """The spec definition in plain Python: unique r in [[T]] with r == z (mod 2^n).""" + r = z % (1 << to.bits) + if to.signed and r >= 1 << (to.bits - 1): + r -= 1 << to.bits + return r + + +def check_wrap_matches_mod_concrete(): + import random + + random.seed(0) + for fn, tn in WRAP_PAIRS: + f, t = TYPES[fn], TYPES[tn] + if f.bits == 8: + samples = list(range(f.min, f.max + 1)) + how = "exhaustive" + else: + samples = {f.min, f.min + 1, -1 if f.signed else 2, 0, 1, f.max - 1, f.max} + samples |= {random.randrange(f.min, f.max + 1) for _ in range(4096)} + samples = sorted(samples) + how = f"{len(samples)} samples" + bad = None + for z in samples: + term = simplify(wrap_int_to_int(BitVecVal(z, f.bits), f, t)) + got = term.as_signed_long() if t.signed else term.as_long() + want = py_wrap(z, t) + if got != want: + bad = (z, got, want) + break + if bad is None: + record(PASS, f"wrap == mod 2^n concrete {fn}->{tn}", how) + else: + record( + FAIL, + f"wrap == mod 2^n concrete {fn}->{tn}", + f"z={bad[0]}: term gives {bad[1]}, spec says {bad[2]}", + ) + + +# --- T7: double rounding --------------------------------------------------------- + + +def check_double_rounding(): + f32, f64 = TYPES["F32"], TYPES["F64"] + # A witness must exist that direct U64->F32 differs from U64->F64->F32 + # (this is the sanity check that the harness can see double rounding at all) + u64 = TYPES["U64"] + v = Const("dr_v", u64.sort) + direct = int_to_float(v, u64, f32) + via_f64 = float_to_float(int_to_float(v, u64, f64), f64, f32) + find( + "T7 direct U64->F32 != U64->F64->F32 (expected witness)", direct != via_f64, [v] + ) + + # ... but for I32 sources the two coincide (I32 -> F64 is exact, + # so only one true rounding happens) + i32 = TYPES["I32"] + w = Const("dr_w", i32.sort) + direct32 = int_to_float(w, i32, f32) + via32 = float_to_float(int_to_float(w, i32, f64), f64, f32) + prove("T7 direct I32->F32 == I32->F64->F32", direct32 == via32) + + +def main(): + print("fpy cast specification checks (see MATH_CASTS_DRAFT.md)\n") + check_totality_note() + check_identity_note() + check_well_formed() + check_tosbv_guards_sufficient() + check_model_independence() + check_saturation_boundaries() + check_sign_of_zero_narrowing() + check_round_trips() + check_wrap_matches_mod() + check_wrap_matches_mod_concrete() + check_double_rounding() + check_monotonicity() + + n_pass = sum(1 for s, _, _ in results if s == PASS) + n_fail = sum(1 for s, _, _ in results if s == FAIL) + n_warn = sum(1 for s, _, _ in results if s == WARN) + print(f"\n{n_pass} passed, {n_fail} failed, {n_warn} unknown/timeout") + sys.exit(1 if n_fail else 0) + + +if __name__ == "__main__": + main() diff --git a/verify/check_llvm_casts.py b/verify/check_llvm_casts.py new file mode 100644 index 0000000..786eb58 --- /dev/null +++ b/verify/check_llvm_casts.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +"""Prove that fpy's LLVM cast codegen implements the MATH_CASTS_DRAFT.md spec. + +For every ordered pair (S, T) of the 10 numeric types, this script: + + 1. calls the *production* codegen (EmitLlvmExpr.convert_numeric_type in + src/fpy/codegen_llvm.py) to emit a one-cast LLVM function S -> T, + 2. round-trips the module text through LLVM's own parser/verifier and + denotes the function as a Z3 term using verify/llvm_semantics.py + (instruction semantics transcribed from Alive2 -- an encoding of LLVM + independent of the spec's), + 3. proves the denotation equal to the spec function cast() from + verify/cast_properties.py for all inputs. + +A PASS on a pair is a machine-checked theorem: "the IR fpy emits for this +cast computes exactly the function the spec defines" -- modulo the shared +FP-theory blind spot for NaN payloads (see llvm_semantics.py docstring). + +Run: + uv run --with z3-solver python verify/check_llvm_casts.py +""" + +import sys +from itertools import product +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from llvmlite import ir + +from fpy import types as fpy_types +from fpy.codegen_llvm import EmitLlvmExpr + +import cast_properties as spec +from llvm_semantics import denote_module + + +def emit_cast_module() -> str: + """One LLVM function per (S, T) pair, bodies produced by fpy's codegen.""" + module = ir.Module(name="fpy_casts") + names = list(spec.TYPES) + for from_name, to_name in product(names, names): + from_fpy = getattr(fpy_types, from_name) + to_fpy = getattr(fpy_types, to_name) + fnty = ir.FunctionType(to_fpy.llvm_type, [from_fpy.llvm_type]) + fn = ir.Function(module, fnty, name=f"cast_{from_name}_{to_name}") + fn.args[0].name = "x" + builder = ir.IRBuilder(fn.append_basic_block("entry")) + result = EmitLlvmExpr(builder).convert_numeric_type( + fn.args[0], from_fpy, to_fpy + ) + builder.ret(result) + return str(module) + + +def main(): + print("fpy LLVM cast codegen vs MATH_CASTS_DRAFT.md spec\n") + + denotations = denote_module(emit_cast_module()) + + names = list(spec.TYPES) + for from_name, to_name in product(names, names): + frm, to = spec.TYPES[from_name], spec.TYPES[to_name] + args, impl = denotations[f"cast_{from_name}_{to_name}"] + assert len(args) == 1 + x = args[0] + assert x.sort() == frm.sort, (x.sort(), frm.sort) + assert impl.sort() == to.sort, (impl.sort(), to.sort) + # SMT `=`: on bitvectors exact, on floats it distinguishes +0/-0 and + # is true on NaN==NaN, i.e. equality of spec values. + spec.prove( + f"codegen == spec {from_name}->{to_name}", impl == spec.cast(x, frm, to) + ) + + n_pass = sum(1 for s, _, _ in spec.results if s == spec.PASS) + n_fail = sum(1 for s, _, _ in spec.results if s == spec.FAIL) + n_warn = sum(1 for s, _, _ in spec.results if s == spec.WARN) + print(f"\n{n_pass} passed, {n_fail} failed, {n_warn} unknown/timeout") + sys.exit(1 if n_fail else 0) + + +if __name__ == "__main__": + main() diff --git a/verify/llvm_semantics.py b/verify/llvm_semantics.py new file mode 100644 index 0000000..7f5a488 --- /dev/null +++ b/verify/llvm_semantics.py @@ -0,0 +1,275 @@ +"""Z3 denotations of the LLVM IR instructions emitted by fpy's cast codegen. + +Each encoder maps one LLVM instruction to a Z3 term over the SMT bitvector +and IEEE-754 floating-point theories. The encodings are transcribed from +Alive2 (https://github.com/AliveToolkit/alive2, commit 1d1bc4f), which is the +de facto SMT semantics of LLVM IR: + + * int conversions: ir/instr.cpp ConversionOp::toSMT + * float conversions: ir/instr.cpp FpConversionOp::toSMT + * Z3-level ops: smt/expr.cpp fp2sint / fp2uint / sint2fp / uint2fp / + float2Float / round + +The transcription is deliberately *independent* of the spec functions in +cast_properties.py: notably, Alive2 detects float->int overflow with an +RTZ round-trip check (sint2fp(fp2sint(x)) == round(x)) instead of the spec's +bound comparisons, so proving the two equal is a real theorem, not an +identity. + +Denotational scope (asserted, not silently ignored): straight-line functions +of one basic block, all instructions total and non-poison -- exactly the +fragment fpy's convert_numeric_type emits. Extending to branching/trapping +code (Tier 3: arithmetic overflow checks) will require path conditions and a +(value, halted) result pair. + +Known blind spot, shared with cast_properties.py: Z3's FP sort has a single +NaN value, so NaN *payloads* are outside this model. LLVM leaves the payload +of NaN results loosely specified, so the spec's canonical-NaN clause cannot +be verified here (Alive2 models floats as bitvectors precisely for this +reason). See MATH_CASTS_DRAFT.md. +""" + +from z3 import ( + RNE, + RTZ, + BitVecSort, + BitVecVal, + Const, + Extract, + FPSort, + FPVal, + If, + Or, + SignExt, + ZeroExt, + fpIsNaN, + fpIsNegative, + fpIsZero, + fpRoundToIntegral, + fpSignedToFP, + fpToFP, + fpToSBV, + fpToUBV, + fpUnsignedToFP, + is_bv_sort, +) + +import llvmlite.binding as llvm_binding + +F32S = FPSort(8, 24) +F64S = FPSort(11, 53) + +_SORTS = { + "i1": BitVecSort(1), + "i8": BitVecSort(8), + "i16": BitVecSort(16), + "i32": BitVecSort(32), + "i64": BitVecSort(64), + "float": F32S, + "double": F64S, +} + + +def sort_of(llvm_type_str: str): + assert llvm_type_str in _SORTS, f"unsupported LLVM type: {llvm_type_str}" + return _SORTS[llvm_type_str] + + +# --- instruction encoders ------------------------------------------------------ +# One function per opcode. Signature: (operand z3 exprs..., result sort) -> expr. + + +def encode_trunc(v, to_sort): + # alive2 ConversionOp::toSMT, Trunc: val.trunc(bits) + return Extract(to_sort.size() - 1, 0, v) + + +def encode_zext(v, to_sort): + # alive2 ConversionOp::toSMT, ZExt: val.zext(delta) + return ZeroExt(to_sort.size() - v.size(), v) + + +def encode_sext(v, to_sort): + # alive2 ConversionOp::toSMT, SExt: val.sext(delta) + return SignExt(to_sort.size() - v.size(), v) + + +def encode_sitofp(v, to_sort): + # alive2 FpConversionOp::toSMT, SIntToFP: val.sint2fp(dummy, rm), rm = RNE + return fpSignedToFP(RNE(), v, to_sort) + + +def encode_uitofp(v, to_sort): + # alive2 FpConversionOp::toSMT, UIntToFP: val.uint2fp(dummy, rm), rm = RNE + # (fpy never sets the nneg flag, so no poison condition) + return fpUnsignedToFP(RNE(), v, to_sort) + + +def encode_fpext(v, to_sort): + # alive2 FpConversionOp::toSMT, FPExt/FPTrunc: val.float2Float(dummy, rm) + # = Z3_mk_fpa_to_fp_float with rm = RNE. fpy emits no fast-math flags, so + # alive2's fm_poison wrapper contributes nothing. + return fpToFP(RNE(), v, to_sort) + + +encode_fptrunc = encode_fpext + + +# for all F32s, we want to show that we produce a value I32 and that value is the +# largest I32 with magnitude less than the F32 +# -- what if val is > I32_MAX? +# -- what if val is NaN? +# -- what if val < I32_MIN? + +# val: F32 = x +# I32(val) + +# + + +def encode_fptosi_sat(v, to_sort): + """alive2 FpConversionOp::toSMT, FPToSInt_Sat. + + Overflow is detected by an RTZ round-trip instead of bound comparisons: + bv is trusted iff sint2fp(bv) reproduces round(v, RTZ). fp.to_sbv is + underspecified out of range, but then no in-range bv round-trips onto the + (out-of-range) rounded value, so every interpretation falls through to + the saturation arms. + """ + bits = to_sort.size() + rm = RTZ() + bv = fpToSBV(rm, v, to_sort) # val.fp2sint(bits, rm) + fp2 = fpSignedToFP(rm, bv, v.sort()) # bv.sint2fp(val, rm) + val_rounded = fpRoundToIntegral(rm, v) # val.round(rm) + # "-0.xx is converted to 0 and then to 0.0, though -0.xx is ok to convert" + # (fp2 is +0 while val_rounded is -0, and SMT `=` distinguishes them) + no_overflow = Or(fpIsZero(val_rounded), fp2 == val_rounded) + return If( + fpIsNaN(v), + BitVecVal(0, bits), + If( + no_overflow, + bv, + If( + fpIsNegative(v), + BitVecVal(-(1 << (bits - 1)), bits), # expr::IntSMin + BitVecVal((1 << (bits - 1)) - 1, bits), # expr::IntSMax + ), + ), + ) + + +def encode_fptoui_sat(v, to_sort): + """alive2 FpConversionOp::toSMT, FPToUInt_Sat.""" + bits = to_sort.size() + rm = RTZ() + bv = fpToUBV(rm, v, to_sort) # val.fp2uint(bits, rm) + fp2 = fpUnsignedToFP(rm, bv, v.sort()) # bv.uint2fp(val, rm) + val_rounded = fpRoundToIntegral(rm, v) # val.round(rm) + # "-0.xx must be converted to 0, not poison." + no_overflow = Or(fpIsZero(val_rounded), fp2 == val_rounded) + return If( + Or(fpIsNaN(v), fpIsNegative(v)), + BitVecVal(0, bits), + If(no_overflow, bv, BitVecVal((1 << bits) - 1, bits)), # expr::IntUMax + ) + + +_SAT_INTRINSICS = { + "llvm.fptosi.sat": encode_fptosi_sat, + "llvm.fptoui.sat": encode_fptoui_sat, +} + +_OPCODE_ENCODERS = { + "trunc": encode_trunc, + "zext": encode_zext, + "sext": encode_sext, + "sitofp": encode_sitofp, + "uitofp": encode_uitofp, + "fpext": encode_fpext, + "fptrunc": encode_fptrunc, +} + + +# --- the symbolic executor ----------------------------------------------------- + + +def _constant_expr(operand): + sort = sort_of(str(operand.type)) + val = operand.get_constant_value() + if is_bv_sort(sort): + return BitVecVal(val, sort.size()) + return FPVal(val, sort) + + +def denote_function(fn): + """Denote a parsed straight-line LLVM function as a Z3 term. + + `fn` is an llvmlite.binding ValueRef of a defined function. Returns + (args, result): `args` are fresh Z3 constants standing for the function + parameters, `result` is the Z3 expression of the returned value. + """ + env = {} + args = [] + for a in fn.arguments: + assert a.name, f"unnamed argument in @{fn.name}" + c = Const(f"{fn.name}!{a.name}", sort_of(str(a.type))) + env[a.name] = c + args.append(c) + + def resolve(operand): + if operand.name: + assert ( + operand.name in env + ), f"@{fn.name}: operand %{operand.name} used before definition" + return env[operand.name] + return _constant_expr(operand) + + blocks = list(fn.blocks) + assert len(blocks) == 1, ( + f"@{fn.name}: denotation only covers straight-line code, " + f"got {len(blocks)} basic blocks" + ) + + result = None + for inst in blocks[0].instructions: + opcode = inst.opcode + operands = list(inst.operands) + + if opcode == "ret": + assert len(operands) == 1, f"@{fn.name}: void or multi-value ret" + assert result is None + result = resolve(operands[0]) + elif opcode == "call": + callee = operands[-1].name + base = ".".join(callee.split(".")[:3]) # llvm.fptosi.sat.i8.f64 -> + assert base in _SAT_INTRINSICS, f"@{fn.name}: unsupported call to @{callee}" + assert len(operands) == 2, f"@{fn.name}: sat intrinsics take one arg" + assert inst.name, f"@{fn.name}: unnamed instruction result" + env[inst.name] = _SAT_INTRINSICS[base]( + resolve(operands[0]), sort_of(str(inst.type)) + ) + else: + assert ( + opcode in _OPCODE_ENCODERS + ), f"@{fn.name}: unsupported opcode {opcode}" + assert len(operands) == 1 + assert inst.name, f"@{fn.name}: unnamed instruction result" + env[inst.name] = _OPCODE_ENCODERS[opcode]( + resolve(operands[0]), sort_of(str(inst.type)) + ) + + assert result is not None, f"@{fn.name}: no ret instruction" + return args, result + + +def denote_module(ll_text: str): + """Parse LLVM assembly (via LLVM itself, which also verifies it) and + denote every defined function. Returns {name: (args, result)}.""" + mod = llvm_binding.parse_assembly(ll_text) + out = {} + for fn in mod.functions: + if fn.is_declaration: + continue + out[fn.name] = denote_function(fn) + return out diff --git a/verify/spec_links.py b/verify/spec_links.py index 676cfc4..1627263 100644 --- a/verify/spec_links.py +++ b/verify/spec_links.py @@ -1,22 +1,21 @@ #!/usr/bin/env python3 -"""Verify spec-to-test links in markdown files. +"""Verify spec-to-test links in the AsciiDoc spec sources. The spec references the tests that pin down a statement with compact numbered -markdown links. The link title (shown on hover) names the test, and the URL +AsciiDoc links. The link's title (shown on hover) names the test, and the URL points at its line: - *Tests:* [1](test/fpy/test_imports.py#L123 "test/fpy/test_imports.py::TestFoo::test_bar"), [2](...) + _Tests:_ link:test/fpy/test_imports.py#L123[1,title="test/fpy/test_imports.py::TestFoo::test_bar"], link:...[2,title="..."] -Any markdown link whose title contains `.py::` is treated as a test link. +Any link macro whose title contains `.py::` is treated as a test link. The title must be `::::` (or `::` for -a module-level test), with relative to the markdown file's directory. -The URL must be `#L` where is the line of the test's -`def`. The link label must be the link's 1-based position among the test -links on its markdown line. +a module-level test), with relative to the repo root. The URL must be +`#L` where is the line of the test's `def`. The link text +must be the link's 1-based position among the test links on its line. Usage: - python3 verify/spec_links.py [--fix] [FILE.md ...] + python3 verify/spec_links.py [--fix] [FILE.adoc ...] Without --fix, exits nonzero if any link names a missing file or test, has a stale line number, or is mislabeled. With --fix, stale line numbers and @@ -29,8 +28,10 @@ import sys from pathlib import Path +REPO = Path(__file__).resolve().parent.parent + TEST_LINK_RE = re.compile( - r'\[(?P