diff --git a/docs/Changelog.md b/docs/Changelog.md index 198aa0b73db..a883b1654ae 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -33093,6 +33093,125 @@ This version of the operator has been available since version 28 of the default
Constrain input and output types to float tensors.
+### **RandomUniform-28** + + Generate a tensor with random values drawn from a uniform distribution. The shape + of the tensor is specified by the `shape` argument and the range by `low` and `high`. + + The data type is specified by the 'dtype' argument. The 'dtype' argument must + be one of the data types specified in the 'DataType' enum field in the + TensorProto message. + + The `generator` attribute selects the pseudo-random number generator algorithm. + With the default value "unspecified", the choice of generator is left to the + implementation and no determinism guarantee is given: results may differ across + implementations and even across runs of the same implementation, even when a + seed is specified. An implementation may produce reproducible results in this + mode (for example for a fixed `seed`), but it is not required to. Setting + `generator` to "philox4x32_10" fully specifies the generated values: given + the same `seed_int64`, every conforming implementation must produce bit-identical + results, which makes the operator deterministic and testable. More algorithms + may be added in future opset versions. + + When `generator` is "philox4x32_10", the `seed_int64` attribute must be specified + (the float `seed` attribute must not be used) and the output is computed with + the Philox-4x32 counter-based generator with 10 rounds (Salmon et al., + "Parallel random numbers: as easy as 1, 2, 3", SC'11), using the standard + constants M0 = 0xD2511F53, M1 = 0xCD9E8D57, W0 = 0x9E3779B9, W1 = 0xBB67AE85. + All arithmetic on counter, key, and output words is unsigned 32-bit modular + arithmetic: + 1. The key is the value of `seed_int64` with its two's complement bits interpreted + as an unsigned 64-bit integer: `key0 = seed_int64 & 0xFFFFFFFF` and + `key1 = (seed_int64 >> 32) & 0xFFFFFFFF`. + 2. Counter block `b` (a 64-bit block index) is the 128-bit counter + `(c0, c1, c2, c3) = (b & 0xFFFFFFFF, (b >> 32) & 0xFFFFFFFF, + offset & 0xFFFFFFFF, (offset >> 32) & 0xFFFFFFFF)`, where `offset` is the + value of the optional `offset` input (0 if not provided) with its two's + complement bits interpreted as an unsigned 64-bit integer. The counter is + encrypted to four 32-bit output words `w0, w1, w2, w3` by applying the + Philox round function 10 times with round keys `(k0, k1)`, starting at + `(key0, key1)` and incremented by `(W0, W1)` before every round except the + first. One round maps `(c0, c1, c2, c3)` to + `(hi1 XOR c1 XOR k0, lo1, hi0 XOR c3 XOR k1, lo0)`, where `hi0` and `lo0` + are the high and low 32 bits of the 64-bit product `M0 * c0`, and `hi1` and + `lo1` are those of `M1 * c2`. + 3. Output element `i` (in row-major order) draws a value `r` in the interval + [0, 1) whose resolution matches the precision of `dtype`. Let `p` be the + number of significand bits of `dtype`, including the implicit bit (8 for + bfloat16, 11 for float16, 24 for float, 53 for double): + - If `dtype` is double, element `i` uses words `a = w(2 * (i mod 2))` and + `b = w(2 * (i mod 2) + 1)` of block `floor(i / 2)` and forms + `r = (floor(a / 2^5) * 2^26 + floor(b / 2^6)) / 2^53`. + - Otherwise, element `i` uses word `a = w(i mod 4)` of block `floor(i / 4)` + and forms `r = floor(a / 2^(32-p)) / 2^p`, which is exactly representable + in `dtype`. + 4. The element value is `low + r * (high - low)`, where `low` and `high` are + first converted to `dtype` and the subtraction, multiplication, and + addition are performed in `dtype` with IEEE 754 round-to-nearest-even + semantics. Note that due to this rounding, the result may equal `high` for + low-precision types. + + Because Philox is counter-based, each output element depends only on `seed_int64`, + `offset`, and its position `i`: elements can be computed independently, in any + order, or in parallel. The block index occupies counter words `c0`/`c1` and the + offset occupies `c2`/`c3`, so the streams of different offsets never overlap, + regardless of the output size. + + A model run is a pure function of its inputs: with a constant (or absent) + `offset`, every run draws the same values, which makes the operator testable. + For streaming inference, feed a different `offset` in every run — since every + offset value selects an independent stream, any non-repeating scheme works, + such as a step counter maintained by the host, stored as an initializer and + advanced at checkpoint time, or carried through a Loop and incremented in the + graph. Each run then draws a fresh, disjoint stream while remaining + individually deterministic and replayable. + +#### Version + +This version of the operator has been available since version 28 of the default ONNX operator set. + +#### Attributes + +
+
dtype : int (default is 1)
+
The data type for the elements of the output tensor. If not specified, default is TensorProto::FLOAT.
+
generator : string (default is unspecified)
+
(Optional) The pseudo-random number generator algorithm: "unspecified" leaves the choice of generator to the implementation and provides no determinism guarantee, even when a seed is specified; "philox4x32_10" selects the fully specified Philox-4x32-10 counter-based algorithm described in the operator documentation, making the output deterministic for a given `seed_int64`. More algorithms may be added in future opset versions.
+
high : float (default is 1.0)
+
Upper boundary of the output values.
+
low : float (default is 0.0)
+
Lower boundary of the output values.
+
seed : float
+
(Optional) Seed to the random generator, if not specified we will auto generate one. Used only when `generator` is "unspecified" (with implementation-defined effect); must not be specified together with a deterministic generator, which uses `seed_int64` instead.
+
seed_int64 : int
+
(Optional) 64-bit seed for the fully specified generators; its two's complement bits are interpreted as an unsigned 64-bit integer. Must be specified when `generator` is "philox4x32_10" (the float `seed` attribute is not used in that case). When `generator` is "unspecified", the effect of `seed_int64` is implementation-defined.
+
shape : list of ints (required)
+
The shape of the output tensor.
+
+ +#### Inputs (0 - 1) + +
+
offset (optional) : T2
+
(Optional) Scalar 64-bit stream offset, 0 if not provided. Each offset value selects an independent random stream (see the operator documentation for the exact semantics): feed a different offset in every run (any non-repeating scheme works, e.g. a step counter) to draw fresh, yet reproducible, values per run, or feed a constant (or omit the input) to draw the same values in every run. When `generator` is "unspecified", the effect of `offset` on the generated values is implementation-defined.
+
+ +#### Outputs + +
+
output : T
+
Output tensor of random values drawn from uniform distribution
+
+ +#### Type Constraints + +
+
T : tensor(bfloat16), tensor(float16), tensor(float), tensor(double)
+
Constrain output types to float tensors.
+
T2 : tensor(int64)
+
Constrain the stream offset to int64.
+
+ # ai.onnx.preview ## Version 1 of the 'ai.onnx.preview' operator set ### **ai.onnx.preview.FlexAttention-1** diff --git a/docs/Operators.md b/docs/Operators.md index d7aa4ec05bc..feaf21a31e6 100644 --- a/docs/Operators.md +++ b/docs/Operators.md @@ -113,7 +113,7 @@ For an operator input/output's differentiability, it can be differentiable, |RNN|22, 14, 7, 1| |RandomNormal|22, 1| |RandomNormalLike|22, 1| -|RandomUniform|22, 1| +|RandomUniform|28, 22, 1| |RandomUniformLike|22, 1| |Reciprocal|13, 6, 1| |ReduceMax|20, 18, 13, 12, 11, 1| @@ -27515,29 +27515,101 @@ Other versions of this operator: 1 be one of the data types specified in the 'DataType' enum field in the TensorProto message. + The `generator` attribute selects the pseudo-random number generator algorithm. + With the default value "unspecified", the choice of generator is left to the + implementation and no determinism guarantee is given: results may differ across + implementations and even across runs of the same implementation, even when a + seed is specified. An implementation may produce reproducible results in this + mode (for example for a fixed `seed`), but it is not required to. Setting + `generator` to "philox4x32_10" fully specifies the generated values: given + the same `seed_int64`, every conforming implementation must produce bit-identical + results, which makes the operator deterministic and testable. More algorithms + may be added in future opset versions. + + When `generator` is "philox4x32_10", the `seed_int64` attribute must be specified + (the float `seed` attribute must not be used) and the output is computed with + the Philox-4x32 counter-based generator with 10 rounds (Salmon et al., + "Parallel random numbers: as easy as 1, 2, 3", SC'11), using the standard + constants M0 = 0xD2511F53, M1 = 0xCD9E8D57, W0 = 0x9E3779B9, W1 = 0xBB67AE85. + All arithmetic on counter, key, and output words is unsigned 32-bit modular + arithmetic: + 1. The key is the value of `seed_int64` with its two's complement bits interpreted + as an unsigned 64-bit integer: `key0 = seed_int64 & 0xFFFFFFFF` and + `key1 = (seed_int64 >> 32) & 0xFFFFFFFF`. + 2. Counter block `b` (a 64-bit block index) is the 128-bit counter + `(c0, c1, c2, c3) = (b & 0xFFFFFFFF, (b >> 32) & 0xFFFFFFFF, + offset & 0xFFFFFFFF, (offset >> 32) & 0xFFFFFFFF)`, where `offset` is the + value of the optional `offset` input (0 if not provided) with its two's + complement bits interpreted as an unsigned 64-bit integer. The counter is + encrypted to four 32-bit output words `w0, w1, w2, w3` by applying the + Philox round function 10 times with round keys `(k0, k1)`, starting at + `(key0, key1)` and incremented by `(W0, W1)` before every round except the + first. One round maps `(c0, c1, c2, c3)` to + `(hi1 XOR c1 XOR k0, lo1, hi0 XOR c3 XOR k1, lo0)`, where `hi0` and `lo0` + are the high and low 32 bits of the 64-bit product `M0 * c0`, and `hi1` and + `lo1` are those of `M1 * c2`. + 3. Output element `i` (in row-major order) draws a value `r` in the interval + [0, 1) whose resolution matches the precision of `dtype`. Let `p` be the + number of significand bits of `dtype`, including the implicit bit (8 for + bfloat16, 11 for float16, 24 for float, 53 for double): + - If `dtype` is double, element `i` uses words `a = w(2 * (i mod 2))` and + `b = w(2 * (i mod 2) + 1)` of block `floor(i / 2)` and forms + `r = (floor(a / 2^5) * 2^26 + floor(b / 2^6)) / 2^53`. + - Otherwise, element `i` uses word `a = w(i mod 4)` of block `floor(i / 4)` + and forms `r = floor(a / 2^(32-p)) / 2^p`, which is exactly representable + in `dtype`. + 4. The element value is `low + r * (high - low)`, where `low` and `high` are + first converted to `dtype` and the subtraction, multiplication, and + addition are performed in `dtype` with IEEE 754 round-to-nearest-even + semantics. Note that due to this rounding, the result may equal `high` for + low-precision types. + + Because Philox is counter-based, each output element depends only on `seed_int64`, + `offset`, and its position `i`: elements can be computed independently, in any + order, or in parallel. The block index occupies counter words `c0`/`c1` and the + offset occupies `c2`/`c3`, so the streams of different offsets never overlap, + regardless of the output size. + + A model run is a pure function of its inputs: with a constant (or absent) + `offset`, every run draws the same values, which makes the operator testable. + For streaming inference, feed a different `offset` in every run — since every + offset value selects an independent stream, any non-repeating scheme works, + such as a step counter maintained by the host, stored as an initializer and + advanced at checkpoint time, or carried through a Loop and incremented in the + graph. Each run then draws a fresh, disjoint stream while remaining + individually deterministic and replayable. + #### Version -This version of the operator has been available since version 22 of the default ONNX operator set. +This version of the operator has been available since version 28 of the default ONNX operator set. -Other versions of this operator: 1 +Other versions of this operator: 1, 22 #### Attributes
dtype : int (default is 1)
The data type for the elements of the output tensor. If not specified, default is TensorProto::FLOAT.
+
generator : string (default is unspecified)
+
(Optional) The pseudo-random number generator algorithm: "unspecified" leaves the choice of generator to the implementation and provides no determinism guarantee, even when a seed is specified; "philox4x32_10" selects the fully specified Philox-4x32-10 counter-based algorithm described in the operator documentation, making the output deterministic for a given `seed_int64`. More algorithms may be added in future opset versions.
high : float (default is 1.0)
Upper boundary of the output values.
low : float (default is 0.0)
Lower boundary of the output values.
seed : float
-
(Optional) Seed to the random generator, if not specified we will auto generate one.
+
(Optional) Seed to the random generator, if not specified we will auto generate one. Used only when `generator` is "unspecified" (with implementation-defined effect); must not be specified together with a deterministic generator, which uses `seed_int64` instead.
+
seed_int64 : int
+
(Optional) 64-bit seed for the fully specified generators; its two's complement bits are interpreted as an unsigned 64-bit integer. Must be specified when `generator` is "philox4x32_10" (the float `seed` attribute is not used in that case). When `generator` is "unspecified", the effect of `seed_int64` is implementation-defined.
shape : list of ints (required)
The shape of the output tensor.
-#### Inputs +#### Inputs (0 - 1) +
+
offset (optional) : T2
+
(Optional) Scalar 64-bit stream offset, 0 if not provided. Each offset value selects an independent random stream (see the operator documentation for the exact semantics): feed a different offset in every run (any non-repeating scheme works, e.g. a step counter) to draw fresh, yet reproducible, values per run, or feed a constant (or omit the input) to draw the same values in every run. When `generator` is "unspecified", the effect of `offset` on the generated values is implementation-defined.
+
#### Outputs @@ -27551,9 +27623,260 @@ Other versions of this operator: 1
T : tensor(bfloat16), tensor(float16), tensor(float), tensor(double)
Constrain output types to float tensors.
+
T2 : tensor(int64)
+
Constrain the stream offset to int64.
+#### Examples + +
+randomuniform_philox + +```python +"""Intent: base case for the deterministic generator — default range +[0, 1), default dtype (float32), 12 elements spanning three full +Philox counter blocks. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[3, 4], + seed_int64=42, + generator="philox4x32_10", +) + +y = philox_uniform(42, (3, 4), np.float32) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox", +) +``` + +
+ + +
+randomuniform_philox_bfloat16 + +```python +"""Intent: lowest-precision type — r uses only the top 8 bits of an +output word (p=8) and every value must be exactly representable in +bfloat16. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.BFLOAT16, + shape=[10], + seed_int64=3, + generator="philox4x32_10", +) + +y = philox_uniform(3, (10,), ml_dtypes.bfloat16) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_bfloat16", +) +``` + +
+ + +
+randomuniform_philox_double + +```python +"""Intent: the double path — each element combines two output words +of the same block via the res53 scheme (words 0/1 for even, 2/3 for +odd elements), unlike the one-word-per-element mapping of the other +types. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.DOUBLE, + shape=[2, 4], + seed_int64=123, + generator="philox4x32_10", +) + +y = philox_uniform(123, (2, 4), np.float64) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_double", +) +``` + +
+ + +
+randomuniform_philox_float16 + +```python +"""Intent: reduced-precision type — r uses the top 11 bits of an +output word (p=11) and every value must be exactly representable in +float16. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.FLOAT16, + shape=[10], + seed_int64=7, + generator="philox4x32_10", +) + +y = philox_uniform(7, (10,), np.float16) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_float16", +) +``` + +
+ + +
+randomuniform_philox_low_high + +```python +"""Intent: non-default range — verifies that low + r * (high - low) +is evaluated in the target data type (float32) with the specified +rounding, not in double precision. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=5.0, + high=10.0, + shape=[2, 3], + seed_int64=0, + generator="philox4x32_10", +) + +y = philox_uniform(0, (2, 3), np.float32, low=5.0, high=10.0) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_low_high", +) +``` + +
+ + +
+randomuniform_philox_multi_block + +```python +"""Intent: stress the counter-block logic — 35 elements span nine +Philox blocks, with the last block only partially consumed (35 = 8*4 ++ 3), so incorrect block increments, word ordering, or padding +handling become visible. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[5, 7], + seed_int64=2024, + generator="philox4x32_10", +) + +y = philox_uniform(2024, (5, 7), np.float32) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_multi_block", +) +``` + +
+ + +
+randomuniform_philox_nd_shape + +```python +"""Intent: non-trivial output shape — a 4-D shape with a singleton +dimension and a negative `low` checks that the row-major element +ordering is independent of the tensor's rank and that sign handling +in low + r * (high - low) is correct. (A dynamic output shape is not +expressible for RandomUniform: `shape` is a required attribute and +the operator's only optional input is the stream offset, not a shape +tensor; data-dependent shapes are the domain of RandomUniformLike.) +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=-1.0, + high=1.0, + shape=[2, 3, 1, 5], + seed_int64=11, + generator="philox4x32_10", +) + +y = philox_uniform(11, (2, 3, 1, 5), np.float32, low=-1.0, high=1.0) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_nd_shape", +) +``` + +
+ + +
+randomuniform_philox_offset + +```python +"""Intent: streaming support — the offset input keys counter words +c2/c3, selecting a stream disjoint from offset 0 (and from every +other offset value). Feeding a different offset per run (e.g. a step +counter) draws fresh, yet reproducible, values in every run. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=["offset"], + outputs=["y"], + shape=[2, 3], + seed_int64=42, + generator="philox4x32_10", +) + +offset = np.array(5, dtype=np.int64) +y = philox_uniform(42, (2, 3), np.float32, offset=5) +expect( + node, + inputs=[offset], + outputs=[y], + name="test_randomuniform_philox_offset", +) +``` + +
+ + ### **RandomUniformLike** Generate a tensor with random values drawn from a uniform distribution. diff --git a/docs/TestCoverage.md b/docs/TestCoverage.md index 3073d6dea79..f084a57fec0 100644 --- a/docs/TestCoverage.md +++ b/docs/TestCoverage.md @@ -6,7 +6,7 @@ * [Overall Test Coverage](#overall-test-coverage) # Node Test Coverage ## Summary -Node tests have covered 189/201 (94.03%, 5 generators excluded) common operators. +Node tests have covered 190/202 (94.06%, 4 generators excluded) common operators. Node tests have covered 1/1 (100.00%, 0 generators excluded) experimental operators. @@ -19846,6 +19846,241 @@ expect( +### RandomUniform +There are 8 test cases, listed as following: +
+randomuniform_philox + +```python +"""Intent: base case for the deterministic generator — default range +[0, 1), default dtype (float32), 12 elements spanning three full +Philox counter blocks. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[3, 4], + seed_int64=42, + generator="philox4x32_10", +) + +y = philox_uniform(42, (3, 4), np.float32) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox", +) +``` + +
+
+randomuniform_philox_bfloat16 + +```python +"""Intent: lowest-precision type — r uses only the top 8 bits of an +output word (p=8) and every value must be exactly representable in +bfloat16. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.BFLOAT16, + shape=[10], + seed_int64=3, + generator="philox4x32_10", +) + +y = philox_uniform(3, (10,), ml_dtypes.bfloat16) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_bfloat16", +) +``` + +
+
+randomuniform_philox_double + +```python +"""Intent: the double path — each element combines two output words +of the same block via the res53 scheme (words 0/1 for even, 2/3 for +odd elements), unlike the one-word-per-element mapping of the other +types. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.DOUBLE, + shape=[2, 4], + seed_int64=123, + generator="philox4x32_10", +) + +y = philox_uniform(123, (2, 4), np.float64) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_double", +) +``` + +
+
+randomuniform_philox_float16 + +```python +"""Intent: reduced-precision type — r uses the top 11 bits of an +output word (p=11) and every value must be exactly representable in +float16. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.FLOAT16, + shape=[10], + seed_int64=7, + generator="philox4x32_10", +) + +y = philox_uniform(7, (10,), np.float16) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_float16", +) +``` + +
+
+randomuniform_philox_low_high + +```python +"""Intent: non-default range — verifies that low + r * (high - low) +is evaluated in the target data type (float32) with the specified +rounding, not in double precision. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=5.0, + high=10.0, + shape=[2, 3], + seed_int64=0, + generator="philox4x32_10", +) + +y = philox_uniform(0, (2, 3), np.float32, low=5.0, high=10.0) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_low_high", +) +``` + +
+
+randomuniform_philox_multi_block + +```python +"""Intent: stress the counter-block logic — 35 elements span nine +Philox blocks, with the last block only partially consumed (35 = 8*4 ++ 3), so incorrect block increments, word ordering, or padding +handling become visible. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[5, 7], + seed_int64=2024, + generator="philox4x32_10", +) + +y = philox_uniform(2024, (5, 7), np.float32) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_multi_block", +) +``` + +
+
+randomuniform_philox_nd_shape + +```python +"""Intent: non-trivial output shape — a 4-D shape with a singleton +dimension and a negative `low` checks that the row-major element +ordering is independent of the tensor's rank and that sign handling +in low + r * (high - low) is correct. (A dynamic output shape is not +expressible for RandomUniform: `shape` is a required attribute and +the operator's only optional input is the stream offset, not a shape +tensor; data-dependent shapes are the domain of RandomUniformLike.) +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=-1.0, + high=1.0, + shape=[2, 3, 1, 5], + seed_int64=11, + generator="philox4x32_10", +) + +y = philox_uniform(11, (2, 3, 1, 5), np.float32, low=-1.0, high=1.0) +expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_nd_shape", +) +``` + +
+
+randomuniform_philox_offset + +```python +"""Intent: streaming support — the offset input keys counter words +c2/c3, selecting a stream disjoint from offset 0 (and from every +other offset value). Feeding a different offset per run (e.g. a step +counter) draws fresh, yet reproducible, values in every run. +""" +node = onnx.helper.make_node( + "RandomUniform", + inputs=["offset"], + outputs=["y"], + shape=[2, 3], + seed_int64=42, + generator="philox4x32_10", +) + +offset = np.array(5, dtype=np.int64) +y = philox_uniform(42, (2, 3), np.float32, offset=5) +expect( + node, + inputs=[offset], + outputs=[y], + name="test_randomuniform_philox_offset", +) +``` + +
+ + ### Range There are 4 test cases, listed as following:
@@ -30721,9 +30956,6 @@ expect(node, inputs=[x, y], outputs=[z], name="test_xor_bcast4v4d") ### RandomNormalLike (random generator operator) -### RandomUniform (random generator operator) - - ### RandomUniformLike (random generator operator) diff --git a/onnx/backend/test/case/node/randomuniform.py b/onnx/backend/test/case/node/randomuniform.py new file mode 100644 index 00000000000..0e654644b6b --- /dev/null +++ b/onnx/backend/test/case/node/randomuniform.py @@ -0,0 +1,262 @@ +# Copyright (c) ONNX Project Contributors +# +# SPDX-License-Identifier: Apache-2.0 +from __future__ import annotations + +import ml_dtypes +import numpy as np + +import onnx +from onnx.backend.test.case.base import Base +from onnx.backend.test.case.node import expect + + +def philox_uniform(seed, shape, dtype, low=0.0, high=1.0, offset=0): + """Independent implementation of RandomUniform with generator="philox4x32_10". + + Follows the operator specification: Philox-4x32-10 keyed with the 64-bit + seed, counter block b = (lo32(b), hi32(b), lo32(offset), hi32(offset)), + per-element values in [0, 1) with a resolution matching the precision of + `dtype` (two output words per element for double, one otherwise), and + ``low + r * (high - low)`` evaluated in `dtype`. Kept separate from + onnx.reference so the generated test data cross-checks the reference + implementation. + """ + m0, m1 = 0xD2511F53, 0xCD9E8D57 + w0, w1 = 0x9E3779B9, 0xBB67AE85 + seed = int(seed) & 0xFFFFFFFFFFFFFFFF + key0, key1 = seed & 0xFFFFFFFF, seed >> 32 + offset = int(offset) & 0xFFFFFFFFFFFFFFFF + + def block(b): + c = [b & 0xFFFFFFFF, (b >> 32) & 0xFFFFFFFF, offset & 0xFFFFFFFF, offset >> 32] + k0, k1 = key0, key1 + for r in range(10): + if r > 0: + k0 = (k0 + w0) & 0xFFFFFFFF + k1 = (k1 + w1) & 0xFFFFFFFF + p0 = m0 * c[0] + p1 = m1 * c[2] + c = [ + (p1 >> 32) ^ c[1] ^ k0, + p1 & 0xFFFFFFFF, + (p0 >> 32) ^ c[3] ^ k1, + p0 & 0xFFFFFFFF, + ] + return c + + num = int(np.prod(shape)) + if np.dtype(dtype) == np.float64: + r = [] + for i in range(num): + w = block(i // 2) + a, b = w[2 * (i % 2)], w[2 * (i % 2) + 1] + r.append(((a >> 5) * 67108864.0 + (b >> 6)) / 9007199254740992.0) + else: + p = ml_dtypes.finfo(dtype).nmant + 1 + r = [(block(i // 4)[i % 4] >> (32 - p)) / (1 << p) for i in range(num)] + r = np.array(r, dtype=np.float64).reshape(shape).astype(dtype) + low = np.asarray(low, dtype=dtype) + high = np.asarray(high, dtype=dtype) + return r * (high - low) + low + + +class RandomUniform(Base): + @staticmethod + def export_randomuniform_philox() -> None: + """Intent: base case for the deterministic generator — default range + [0, 1), default dtype (float32), 12 elements spanning three full + Philox counter blocks. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[3, 4], + seed_int64=42, + generator="philox4x32_10", + ) + + y = philox_uniform(42, (3, 4), np.float32) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox", + ) + + @staticmethod + def export_randomuniform_philox_multi_block() -> None: + """Intent: stress the counter-block logic — 35 elements span nine + Philox blocks, with the last block only partially consumed (35 = 8*4 + + 3), so incorrect block increments, word ordering, or padding + handling become visible. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + shape=[5, 7], + seed_int64=2024, + generator="philox4x32_10", + ) + + y = philox_uniform(2024, (5, 7), np.float32) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_multi_block", + ) + + @staticmethod + def export_randomuniform_philox_nd_shape() -> None: + """Intent: non-trivial output shape — a 4-D shape with a singleton + dimension and a negative `low` checks that the row-major element + ordering is independent of the tensor's rank and that sign handling + in low + r * (high - low) is correct. (A dynamic output shape is not + expressible for RandomUniform: `shape` is a required attribute and + the operator's only optional input is the stream offset, not a shape + tensor; data-dependent shapes are the domain of RandomUniformLike.) + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=-1.0, + high=1.0, + shape=[2, 3, 1, 5], + seed_int64=11, + generator="philox4x32_10", + ) + + y = philox_uniform(11, (2, 3, 1, 5), np.float32, low=-1.0, high=1.0) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_nd_shape", + ) + + @staticmethod + def export_randomuniform_philox_low_high() -> None: + """Intent: non-default range — verifies that low + r * (high - low) + is evaluated in the target data type (float32) with the specified + rounding, not in double precision. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + low=5.0, + high=10.0, + shape=[2, 3], + seed_int64=0, + generator="philox4x32_10", + ) + + y = philox_uniform(0, (2, 3), np.float32, low=5.0, high=10.0) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_low_high", + ) + + @staticmethod + def export_randomuniform_philox_double() -> None: + """Intent: the double path — each element combines two output words + of the same block via the res53 scheme (words 0/1 for even, 2/3 for + odd elements), unlike the one-word-per-element mapping of the other + types. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.DOUBLE, + shape=[2, 4], + seed_int64=123, + generator="philox4x32_10", + ) + + y = philox_uniform(123, (2, 4), np.float64) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_double", + ) + + @staticmethod + def export_randomuniform_philox_bfloat16() -> None: + """Intent: lowest-precision type — r uses only the top 8 bits of an + output word (p=8) and every value must be exactly representable in + bfloat16. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.BFLOAT16, + shape=[10], + seed_int64=3, + generator="philox4x32_10", + ) + + y = philox_uniform(3, (10,), ml_dtypes.bfloat16) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_bfloat16", + ) + + @staticmethod + def export_randomuniform_philox_offset() -> None: + """Intent: streaming support — the offset input keys counter words + c2/c3, selecting a stream disjoint from offset 0 (and from every + other offset value). Feeding a different offset per run (e.g. a step + counter) draws fresh, yet reproducible, values in every run. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=["offset"], + outputs=["y"], + shape=[2, 3], + seed_int64=42, + generator="philox4x32_10", + ) + + offset = np.array(5, dtype=np.int64) + y = philox_uniform(42, (2, 3), np.float32, offset=5) + expect( + node, + inputs=[offset], + outputs=[y], + name="test_randomuniform_philox_offset", + ) + + @staticmethod + def export_randomuniform_philox_float16() -> None: + """Intent: reduced-precision type — r uses the top 11 bits of an + output word (p=11) and every value must be exactly representable in + float16. + """ + node = onnx.helper.make_node( + "RandomUniform", + inputs=[], + outputs=["y"], + dtype=onnx.TensorProto.FLOAT16, + shape=[10], + seed_int64=7, + generator="philox4x32_10", + ) + + y = philox_uniform(7, (10,), np.float16) + expect( + node, + inputs=[], + outputs=[y], + name="test_randomuniform_philox_float16", + ) diff --git a/onnx/backend/test/data/node/test_randomuniform_philox/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox/model.onnx new file mode 100644 index 00000000000..b76703e0dfe Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..b31a7225a8b --- /dev/null +++ b/onnx/backend/test/data/node/test_randomuniform_philox/test_data_set_0/output_0.pb @@ -0,0 +1 @@ +ByJ0?>=f>!|?t>Z? >lS?](?mM? F? \ No newline at end of file diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/model.onnx new file mode 100644 index 00000000000..8945538576f Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..8f10379b59b --- /dev/null +++ b/onnx/backend/test/data/node/test_randomuniform_philox_bfloat16/test_data_set_0/output_0.pb @@ -0,0 +1,2 @@ + +ByJQ?='?C?> ?d>?D?I? \ No newline at end of file diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_double/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_double/model.onnx new file mode 100644 index 00000000000..908266b8b20 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_double/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_double/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_double/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..1b972d8f9a2 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_double/test_data_set_0/output_0.pb differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_float16/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_float16/model.onnx new file mode 100644 index 00000000000..422daeb4f90 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_float16/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_float16/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_float16/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..243846e3fb3 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_float16/test_data_set_0/output_0.pb differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_low_high/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_low_high/model.onnx new file mode 100644 index 00000000000..e73d985d002 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_low_high/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_low_high/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_low_high/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..3cb00bf9bd3 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_low_high/test_data_set_0/output_0.pb differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/model.onnx new file mode 100644 index 00000000000..8babbcfc854 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..4b1dcb1db26 --- /dev/null +++ b/onnx/backend/test/data/node/test_randomuniform_philox_multi_block/test_data_set_0/output_0.pb @@ -0,0 +1,2 @@ +ByJk? >|?`4=(L>->e?>D?`I=tT?D= +*?>lq>Hm>#-?! Y??0Y>r>`<? >"??H>sS?F?T?=8=w? \ No newline at end of file diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/model.onnx new file mode 100644 index 00000000000..d7d644f1336 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..59e13fa03b3 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_nd_shape/test_data_set_0/output_0.pb differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_offset/model.onnx b/onnx/backend/test/data/node/test_randomuniform_philox_offset/model.onnx new file mode 100644 index 00000000000..1d2ffcdab2e Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_offset/model.onnx differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/input_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/input_0.pb new file mode 100644 index 00000000000..395ab76dce3 Binary files /dev/null and b/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/input_0.pb differ diff --git a/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/output_0.pb new file mode 100644 index 00000000000..abd31818181 --- /dev/null +++ b/onnx/backend/test/data/node/test_randomuniform_philox_offset/test_data_set_0/output_0.pb @@ -0,0 +1 @@ +ByJ>8o>PD>Z_?+q?F? \ No newline at end of file diff --git a/onnx/defs/doc_strings.cc b/onnx/defs/doc_strings.cc index abab3015f08..1adba6b4345 100644 --- a/onnx/defs/doc_strings.cc +++ b/onnx/defs/doc_strings.cc @@ -154,6 +154,79 @@ be one of the data types specified in the 'DataType' enum field in the TensorProto message. )DOC"; +const char kDoc_RandomUniform_ver28[] = R"DOC( +Generate a tensor with random values drawn from a uniform distribution. The shape +of the tensor is specified by the `shape` argument and the range by `low` and `high`. + +The data type is specified by the 'dtype' argument. The 'dtype' argument must +be one of the data types specified in the 'DataType' enum field in the +TensorProto message. + +The `generator` attribute selects the pseudo-random number generator algorithm. +With the default value "unspecified", the choice of generator is left to the +implementation and no determinism guarantee is given: results may differ across +implementations and even across runs of the same implementation, even when a +seed is specified. An implementation may produce reproducible results in this +mode (for example for a fixed `seed`), but it is not required to. Setting +`generator` to "philox4x32_10" fully specifies the generated values: given +the same `seed_int64`, every conforming implementation must produce bit-identical +results, which makes the operator deterministic and testable. More algorithms +may be added in future opset versions. + +When `generator` is "philox4x32_10", the `seed_int64` attribute must be specified +(the float `seed` attribute must not be used) and the output is computed with +the Philox-4x32 counter-based generator with 10 rounds (Salmon et al., +"Parallel random numbers: as easy as 1, 2, 3", SC'11), using the standard +constants M0 = 0xD2511F53, M1 = 0xCD9E8D57, W0 = 0x9E3779B9, W1 = 0xBB67AE85. +All arithmetic on counter, key, and output words is unsigned 32-bit modular +arithmetic: +1. The key is the value of `seed_int64` with its two's complement bits interpreted + as an unsigned 64-bit integer: `key0 = seed_int64 & 0xFFFFFFFF` and + `key1 = (seed_int64 >> 32) & 0xFFFFFFFF`. +2. Counter block `b` (a 64-bit block index) is the 128-bit counter + `(c0, c1, c2, c3) = (b & 0xFFFFFFFF, (b >> 32) & 0xFFFFFFFF, + offset & 0xFFFFFFFF, (offset >> 32) & 0xFFFFFFFF)`, where `offset` is the + value of the optional `offset` input (0 if not provided) with its two's + complement bits interpreted as an unsigned 64-bit integer. The counter is + encrypted to four 32-bit output words `w0, w1, w2, w3` by applying the + Philox round function 10 times with round keys `(k0, k1)`, starting at + `(key0, key1)` and incremented by `(W0, W1)` before every round except the + first. One round maps `(c0, c1, c2, c3)` to + `(hi1 XOR c1 XOR k0, lo1, hi0 XOR c3 XOR k1, lo0)`, where `hi0` and `lo0` + are the high and low 32 bits of the 64-bit product `M0 * c0`, and `hi1` and + `lo1` are those of `M1 * c2`. +3. Output element `i` (in row-major order) draws a value `r` in the interval + [0, 1) whose resolution matches the precision of `dtype`. Let `p` be the + number of significand bits of `dtype`, including the implicit bit (8 for + bfloat16, 11 for float16, 24 for float, 53 for double): + - If `dtype` is double, element `i` uses words `a = w(2 * (i mod 2))` and + `b = w(2 * (i mod 2) + 1)` of block `floor(i / 2)` and forms + `r = (floor(a / 2^5) * 2^26 + floor(b / 2^6)) / 2^53`. + - Otherwise, element `i` uses word `a = w(i mod 4)` of block `floor(i / 4)` + and forms `r = floor(a / 2^(32-p)) / 2^p`, which is exactly representable + in `dtype`. +4. The element value is `low + r * (high - low)`, where `low` and `high` are + first converted to `dtype` and the subtraction, multiplication, and + addition are performed in `dtype` with IEEE 754 round-to-nearest-even + semantics. Note that due to this rounding, the result may equal `high` for + low-precision types. + +Because Philox is counter-based, each output element depends only on `seed_int64`, +`offset`, and its position `i`: elements can be computed independently, in any +order, or in parallel. The block index occupies counter words `c0`/`c1` and the +offset occupies `c2`/`c3`, so the streams of different offsets never overlap, +regardless of the output size. + +A model run is a pure function of its inputs: with a constant (or absent) +`offset`, every run draws the same values, which makes the operator testable. +For streaming inference, feed a different `offset` in every run — since every +offset value selects an independent stream, any non-repeating scheme works, +such as a step counter maintained by the host, stored as an initializer and +advanced at checkpoint time, or carried through a Loop and incremented in the +graph. Each run then draws a fresh, disjoint stream while remaining +individually deterministic and replayable. +)DOC"; + const char kDoc_DequantizeLinear_ver24[] = R"DOC( The linear dequantization operator. It consumes a quantized tensor, a scale, and a zero point to compute the full-precision tensor. The dequantization formula is `y = (x - x_zero_point) * x_scale`. `x_scale` and `x_zero_point` @@ -1318,6 +1391,7 @@ const char kDoc_Squeeze_ver24[] = ""; const char kDoc_MaxUnpool_ver11[] = ""; const char kDoc_Size_ver24[] = ""; const char kDoc_RandomUniform_ver1[] = ""; +const char kDoc_RandomUniform_ver28[] = ""; const char kDoc_Range_ver11[] = ""; const char kDoc_Range_ver27[] = ""; const char kDoc_DequantizeLinear_ver24[] = ""; diff --git a/onnx/defs/doc_strings.h b/onnx/defs/doc_strings.h index 5ebb7875924..ab3d5d7cc87 100644 --- a/onnx/defs/doc_strings.h +++ b/onnx/defs/doc_strings.h @@ -54,6 +54,7 @@ extern const char kDoc_PRelu_ver7[]; extern const char kDoc_RandomNormal_ver1[]; extern const char kDoc_RandomNormalLike_ver1[]; extern const char kDoc_RandomUniform_ver1[]; +extern const char kDoc_RandomUniform_ver28[]; extern const char kDoc_Range_ver11[]; extern const char kDoc_Range_ver27[]; extern const char kDoc_RandomUniformLike_ver1[]; diff --git a/onnx/defs/generator/defs.cc b/onnx/defs/generator/defs.cc index 4272b7b8f8c..3a38fa7ccd4 100644 --- a/onnx/defs/generator/defs.cc +++ b/onnx/defs/generator/defs.cc @@ -150,26 +150,27 @@ ONNX_OPERATOR_SET_SCHEMA( ONNX_OPERATOR_SET_SCHEMA( RandomUniform, - 22, + 28, OpSchema() - .SetDoc(kDoc_RandomUniform_ver1) + .SetDoc(kDoc_RandomUniform_ver28) .Attr("low", "Lower boundary of the output values.", AttributeProto::FLOAT, 0.0f) .Attr("high", "Upper boundary of the output values.", AttributeProto::FLOAT, 1.0f) - .Attr( - "seed", - "(Optional) Seed to the random generator, if not specified we will auto generate one.", - AttributeProto::FLOAT, - OPTIONAL_VALUE) + .Attr("seed", kRandomGeneratorSeedAttrDoc, AttributeProto::FLOAT, OPTIONAL_VALUE) + .Attr("seed_int64", kRandomGeneratorSeedInt64AttrDoc, AttributeProto::INT, OPTIONAL_VALUE) + .Attr("generator", kRandomGeneratorAttrDoc, AttributeProto::STRING, std::string("unspecified")) .Attr( "dtype", "The data type for the elements of the output tensor. If not specified, default is TensorProto::FLOAT.", AttributeProto::INT, static_cast(TensorProto::FLOAT)) .Attr("shape", "The shape of the output tensor.", AttributeProto::INTS) + .Input(0, "offset", kRandomGeneratorOffsetInputDoc, "T2", OpSchema::Optional) .Output(0, "output", "Output tensor of random values drawn from uniform distribution", "T") .TypeConstraint("T", OpSchema::all_float_types_ir4(), "Constrain output types to float tensors.") + .TypeConstraint("T2", {types::Int64}, "Constrain the stream offset to int64.") .SetNodeDeterminism(OpSchema::NodeDeterminism::NonDeterministic) .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { + ValidateRandomGeneratorAttributes(ctx, 0); propagateElemTypeFromAttributeToOutput(ctx, "dtype", 0, TensorProto::FLOAT); propagateShapeFromAttributeToOutput(ctx, "shape", 0); })); diff --git a/onnx/defs/generator/old.cc b/onnx/defs/generator/old.cc index 0d2c356425f..6e50f0d64e2 100644 --- a/onnx/defs/generator/old.cc +++ b/onnx/defs/generator/old.cc @@ -229,6 +229,32 @@ ONNX_OPERATOR_SET_SCHEMA( propagateShapeFromAttributeToOutput(ctx, "shape", 0); })); +ONNX_OPERATOR_SET_SCHEMA( + RandomUniform, + 22, + OpSchema() + .SetDoc(kDoc_RandomUniform_ver1) + .Attr("low", "Lower boundary of the output values.", AttributeProto::FLOAT, 0.0f) + .Attr("high", "Upper boundary of the output values.", AttributeProto::FLOAT, 1.0f) + .Attr( + "seed", + "(Optional) Seed to the random generator, if not specified we will auto generate one.", + AttributeProto::FLOAT, + OPTIONAL_VALUE) + .Attr( + "dtype", + "The data type for the elements of the output tensor. If not specified, default is TensorProto::FLOAT.", + AttributeProto::INT, + static_cast(TensorProto::FLOAT)) + .Attr("shape", "The shape of the output tensor.", AttributeProto::INTS) + .Output(0, "output", "Output tensor of random values drawn from uniform distribution", "T") + .TypeConstraint("T", OpSchema::all_float_types_ir4(), "Constrain output types to float tensors.") + .SetNodeDeterminism(OpSchema::NodeDeterminism::NonDeterministic) + .TypeAndShapeInferenceFunction([](InferenceContext& ctx) { + propagateElemTypeFromAttributeToOutput(ctx, "dtype", 0, TensorProto::FLOAT); + propagateShapeFromAttributeToOutput(ctx, "shape", 0); + })); + ONNX_OPERATOR_SET_SCHEMA( RandomUniform, 1, diff --git a/onnx/defs/generator/utils.cc b/onnx/defs/generator/utils.cc index 8eacc2dbdb3..da12c9efe2a 100644 --- a/onnx/defs/generator/utils.cc +++ b/onnx/defs/generator/utils.cc @@ -108,4 +108,27 @@ void ConstantOpInference(InferenceContext& ctx) { "this line should never be reached."); } +void ValidateRandomGeneratorAttributes(InferenceContext& ctx, int offset_input_index) { + const auto* generator_attr = ctx.getAttribute("generator"); + if (generator_attr != nullptr) { + const std::string& generator = generator_attr->s(); + if (generator != "unspecified" && generator != "philox4x32_10") { + fail_shape_inference( + "Attribute 'generator' must be one of 'unspecified' or 'philox4x32_10', got '", generator, "'."); + } + if (generator != "unspecified") { + if (ctx.getAttribute("seed_int64") == nullptr) { + fail_shape_inference("Attribute 'seed_int64' must be specified when 'generator' is '", generator, "'."); + } + if (ctx.getAttribute("seed") != nullptr) { + fail_shape_inference( + "Attribute 'seed' must not be specified when 'generator' is '", generator, "'; use 'seed_int64' instead."); + } + } + } + if (offset_input_index >= 0) { + checkInputRank(ctx, static_cast(offset_input_index), 0); + } +} + } // namespace ONNX_NAMESPACE diff --git a/onnx/defs/generator/utils.h b/onnx/defs/generator/utils.h index 899a02b682a..1e2cb392175 100644 --- a/onnx/defs/generator/utils.h +++ b/onnx/defs/generator/utils.h @@ -14,6 +14,42 @@ namespace ONNX_NAMESPACE { void ConstantOpInference(InferenceContext& ctx); +// Shared documentation for the deterministic random-generator mechanism +// (generator / seed / seed_int64 attributes and the offset input), introduced +// with RandomUniform-28 and intended to be reused verbatim when the other +// random operators adopt the same mechanism. +inline constexpr const char* kRandomGeneratorSeedAttrDoc = + "(Optional) Seed to the random generator, if not specified we will auto generate one. " + "Used only when `generator` is \"unspecified\" (with implementation-defined effect); must not " + "be specified together with a deterministic generator, which uses `seed_int64` instead."; + +inline constexpr const char* kRandomGeneratorSeedInt64AttrDoc = + "(Optional) 64-bit seed for the fully specified generators; its two's complement bits are " + "interpreted as an unsigned 64-bit integer. Must be specified when `generator` is " + "\"philox4x32_10\" (the float `seed` attribute is not used in that case). When `generator` is " + "\"unspecified\", the effect of `seed_int64` is implementation-defined."; + +inline constexpr const char* kRandomGeneratorAttrDoc = + "(Optional) The pseudo-random number generator algorithm: \"unspecified\" leaves the choice of " + "generator to the implementation and provides no determinism guarantee, even when a seed is " + "specified; \"philox4x32_10\" selects the fully specified Philox-4x32-10 counter-based algorithm " + "described in the operator documentation, making the output deterministic for a given " + "`seed_int64`. More algorithms may be added in future opset versions."; + +inline constexpr const char* kRandomGeneratorOffsetInputDoc = + "(Optional) Scalar 64-bit stream offset, 0 if not provided. Each offset value selects an " + "independent random stream (see the operator documentation for the exact semantics): feed a " + "different offset in every run (any non-repeating scheme works, e.g. a step counter) to draw " + "fresh, yet reproducible, values per run, or feed a constant (or omit the input) to draw the " + "same values in every run. When `generator` is \"unspecified\", the effect of `offset` on the " + "generated values is implementation-defined."; + +// Validates the deterministic random-generator attributes: `generator` must +// be a known algorithm, deterministic generators require `seed_int64` and +// forbid the float `seed`, and the optional offset input (identified by +// `offset_input_index`, or -1 if the operator has none) must be a scalar. +void ValidateRandomGeneratorAttributes(InferenceContext& ctx, int offset_input_index); + template int64_t compute_output_dim_for_range(const TensorProto* start, const TensorProto* limit, const TensorProto* delta) { if (!start->dims().empty() || !limit->dims().empty() || !delta->dims().empty()) { diff --git a/onnx/defs/operator_sets.h b/onnx/defs/operator_sets.h index de677584826..d429656b408 100644 --- a/onnx/defs/operator_sets.h +++ b/onnx/defs/operator_sets.h @@ -1471,12 +1471,14 @@ class OpSet_Onnx_ver27 { // Forward declarations for ai.onnx version 28 class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 28, Celu); +class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 28, RandomUniform); // Iterate over schema from ai.onnx version 28 class OpSet_Onnx_ver28 { public: static void ForEachSchema(const std::function& fn) { fn(GetOpSchema()); + fn(GetOpSchema()); } }; diff --git a/onnx/reference/ops/_op_common_random.py b/onnx/reference/ops/_op_common_random.py index c7b011e2929..6d1f7923999 100644 --- a/onnx/reference/ops/_op_common_random.py +++ b/onnx/reference/ops/_op_common_random.py @@ -3,17 +3,134 @@ # SPDX-License-Identifier: Apache-2.0 from __future__ import annotations +import ml_dtypes import numpy as np from onnx.helper import tensor_dtype_to_np_dtype from onnx.reference.op_run import OpRun +class _Philox4x32: + """Philox-4x32-10 counter-based PRNG. + + Implements the Philox-4x32 generator with 10 rounds and the standard + constants from Salmon et al., "Parallel random numbers: as easy as + 1, 2, 3" (SC'11), as also distributed in the Random123 library. This is + the algorithm selected by the ``generator="philox4x32_10"`` attribute of + the random operators, which fully specifies their output for a given + seed. Being counter-based, every output word depends only on the key + (derived from the seed) and the block index, so elements can be computed + independently and in parallel. + """ + + _M0 = 0xD2511F53 + _M1 = 0xCD9E8D57 + _W0 = 0x9E3779B9 + _W1 = 0xBB67AE85 + + def __init__(self, seed: int, offset: int = 0): + seed &= 0xFFFFFFFFFFFFFFFF + self._key0 = seed & 0xFFFFFFFF + self._key1 = seed >> 32 + # The stream offset occupies counter words c2/c3 (two's complement + # bits interpreted as unsigned), so different offsets select disjoint + # streams regardless of how many blocks are consumed. + offset &= 0xFFFFFFFFFFFFFFFF + self._offset0 = offset & 0xFFFFFFFF + self._offset1 = offset >> 32 + + @classmethod + def philox4x32_10(cls, c0, c1, c2, c3, key0: int, key1: int): + """Encrypt 128-bit counters (four uint32 arrays) with 10 Philox rounds. + + Returns the four 32-bit output words per counter. The round keys start + at ``(key0, key1)`` and are incremented by ``(W0, W1)`` before every + round except the first. + """ + mask = np.uint64(0xFFFFFFFF) + c0 = np.asarray(c0, dtype=np.uint64) + c1 = np.asarray(c1, dtype=np.uint64) + c2 = np.asarray(c2, dtype=np.uint64) + c3 = np.asarray(c3, dtype=np.uint64) + k0, k1 = key0, key1 + for r in range(10): + if r > 0: + k0 = (k0 + cls._W0) & 0xFFFFFFFF + k1 = (k1 + cls._W1) & 0xFFFFFFFF + p0 = np.uint64(cls._M0) * c0 + p1 = np.uint64(cls._M1) * c2 + c0, c1, c2, c3 = ( + (p1 >> np.uint64(32)) ^ c1 ^ np.uint64(k0), + p1 & mask, + (p0 >> np.uint64(32)) ^ c3 ^ np.uint64(k1), + p0 & mask, + ) + return ( + c0.astype(np.uint32), + c1.astype(np.uint32), + c2.astype(np.uint32), + c3.astype(np.uint32), + ) + + def _words(self, num: int, words_per_element: int) -> np.ndarray: + """Output words of enough counter blocks for `num` elements. + + Block ``b`` uses the counter ``(lo32(b), hi32(b), lo32(offset), + hi32(offset))``. Returns the words as an array of shape + ``(num_blocks, 4)`` in block order. + """ + num_blocks = (num * words_per_element + 3) // 4 + b = np.arange(num_blocks, dtype=np.uint64) + return np.stack( + self.philox4x32_10( + b & np.uint64(0xFFFFFFFF), + b >> np.uint64(32), + np.uint64(self._offset0), + np.uint64(self._offset1), + self._key0, + self._key1, + ), + axis=1, + ) + + def random_res53(self, num: int) -> np.ndarray: + """Draw `num` doubles in [0, 1) with 53-bit resolution. + + Element `i` combines words ``2*(i mod 2)`` and ``2*(i mod 2) + 1`` of + block ``i // 2`` as ``(floor(a / 2^5) * 2^26 + floor(b / 2^6)) / 2^53``. + """ + w = self._words(num, 2) + a = w[:, [0, 2]].reshape(-1)[:num] >> np.uint32(5) + b = w[:, [1, 3]].reshape(-1)[:num] >> np.uint32(6) + return (a.astype(np.float64) * 67108864.0 + b.astype(np.float64)) * ( + 1.0 / 9007199254740992.0 + ) + + def random_res(self, num: int, precision: int) -> np.ndarray: + """Draw `num` values in [0, 1) with `precision` significand bits. + + Element `i` uses word ``i mod 4`` of block ``i // 4``: + ``(w >> (32 - p)) / 2^p``. Each value has at most 24 significand + bits, so the float32 result is exact and representable in any binary + float type with at least `precision` significand bits. + """ + words = self._words(num, 1).reshape(-1)[:num] + scale = np.float32(1.0 / (1 << precision)) + return (words >> np.uint32(32 - precision)).astype(np.float32) * scale + + class _CommonRandom(OpRun): def __init__(self, onnx_node, run_params): OpRun.__init__(self, onnx_node, run_params) - if hasattr(self, "shape") and len(self.shape) == 0: - raise ValueError( # pragma: no cover + if ( + hasattr(self, "shape") + and len(self.shape) == 0 + # An empty shape (scalar output) is fully specified for the + # deterministic generators; only the legacy "unspecified" path + # of this implementation does not support it. + and getattr(self, "generator", None) in (None, "unspecified") + ): + raise ValueError( f"shape cannot be empty for operator {self.__class__.__name__}." ) @@ -53,3 +170,47 @@ def _get_state(seed): else: state = np.random.RandomState(seed=int(seed)) return state + + @staticmethod + def _deterministic_uniform( + generator, seed, seed_int64, shape, dtype, low, high, offset + ): + """Compute the fully specified deterministic uniform output. + + Validates the generator attributes, draws values in [0, 1) with a + resolution matching the precision of `dtype` (double combines two + 32-bit output words per element, all other float types use one word + per element, keeping every value exactly representable in `dtype`), + and evaluates ``low + r * (high - low)`` in `dtype`. Unlike the + "unspecified" generator, the result is bit-identical across + implementations for a given seed_int64 and offset (see the operator + specification). + """ + if generator != "philox4x32_10": + raise ValueError( + f"Unsupported value {generator!r} for attribute 'generator'." + ) + if seed_int64 is None: + raise ValueError( + "Attribute 'seed_int64' must be specified when 'generator' is " + f"{generator!r}." + ) + if seed is not None: + raise ValueError( + "Attribute 'seed' must not be specified when 'generator' is " + f"{generator!r}; use 'seed_int64' instead." + ) + offset_value = 0 if offset is None else int(np.asarray(offset).item()) + state = _Philox4x32(int(seed_int64), offset_value) + num = int(np.prod(shape)) + if np.dtype(dtype) == np.float64: + res = state.random_res53(num) + else: + # ml_dtypes.finfo also covers non-native types such as bfloat16 + precision = ml_dtypes.finfo(dtype).nmant + 1 + res = state.random_res(num, precision) + res = res.reshape(shape).astype(dtype, copy=False) + # low + r * (high - low), evaluated in the target data type + low_t = np.asarray(low, dtype=dtype) + high_t = np.asarray(high, dtype=dtype) + return res * (high_t - low_t) + low_t diff --git a/onnx/reference/ops/op_random_uniform.py b/onnx/reference/ops/op_random_uniform.py index be6a74b3ac2..bdf026c0f02 100644 --- a/onnx/reference/ops/op_random_uniform.py +++ b/onnx/reference/ops/op_random_uniform.py @@ -7,8 +7,26 @@ class RandomUniform(_CommonRandom): - def _run(self, dtype=None, high=None, low=None, seed=None, shape=None): + def _run( + self, + offset=None, + dtype=None, + generator=None, + high=None, + low=None, + seed=None, + seed_int64=None, + shape=None, + ): dtype = self._dtype(dtype=dtype) + if generator not in (None, "unspecified"): + return ( + self._deterministic_uniform( + generator, seed, seed_int64, shape, dtype, low, high, offset + ), + ) + # The effect of offset on the values is implementation-defined for + # the "unspecified" generator; it is ignored here. state = self._get_state(seed) res = state.rand(*shape).astype(dtype) res *= high - low diff --git a/onnx/test/reference_evaluator_test.py b/onnx/test/reference_evaluator_test.py index c2cefbca4c9..0a02f8060f7 100644 --- a/onnx/test/reference_evaluator_test.py +++ b/onnx/test/reference_evaluator_test.py @@ -1477,6 +1477,249 @@ def test_onnxt_runtime_random_uniform(self): self.assertGreater(got.min(), 0) self.assertLess(got.max(), 1) + def test_onnxt_runtime_random_uniform_philox(self): + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) + node1 = make_node( + "RandomUniform", + [], + ["Y"], + seed_int64=42, + shape=[2, 3], + generator="philox4x32_10", + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + sess = ReferenceEvaluator(onnx_model) + got = sess.run(None, {})[0] + # For float32, element i uses word (i mod 4) of Philox-4x32-10 block + # (i // 4) with key (42, 0): r = (w >> 8) / 2^24. Word stream produced + # by the canonical Random123 implementation. + expected = np.array( + [ + [0.61295986, 0.4685865, 0.0732317], + [0.3408615, 0.98771864, 0.32706332], + ], + dtype=np.float32, + ) + assert_allclose(got, expected, rtol=0, atol=0) + self.assertEqual(got.dtype, np.float32) + # A second run must produce bit-identical values. + assert_allclose(sess.run(None, {})[0], expected, rtol=0, atol=0) + + def test_onnxt_runtime_random_uniform_philox_low_high(self): + Y = make_tensor_value_info("Y", TensorProto.DOUBLE, [None]) + node1 = make_node( + "RandomUniform", + [], + ["Y"], + seed_int64=42, + low=5.0, + high=10.0, + dtype=TensorProto.DOUBLE, + shape=[3], + generator="philox4x32_10", + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + sess = ReferenceEvaluator(onnx_model) + got = sess.run(None, {})[0] + # For double, element i combines words 2*(i mod 2) and 2*(i mod 2)+1 + # of Philox-4x32-10 block (i // 2) with key (42, 0) via + # r = ((a >> 5) * 2^26 + (b >> 6)) / 2^53. + expected = 5.0 + np.array( + [0.6129598801477738, 0.07323173687503892, 0.9877186516453577], + dtype=np.float64, + ) * (10.0 - 5.0) + assert_allclose(got, expected, rtol=0, atol=0) + self.assertEqual(got.dtype, np.float64) + + def test_onnxt_runtime_random_uniform_philox_bfloat16(self): + Y = make_tensor_value_info("Y", TensorProto.BFLOAT16, [None]) + node1 = make_node( + "RandomUniform", + [], + ["Y"], + seed_int64=3, + dtype=TensorProto.BFLOAT16, + shape=[4], + generator="philox4x32_10", + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + sess = ReferenceEvaluator(onnx_model) + got = sess.run(None, {})[0] + # For bfloat16 (p=8), element i uses word (i mod 4) of Philox-4x32-10 + # block (i // 4) with key (3, 0): r = (w >> 24) / 2^8. Word stream + # produced by the canonical Random123 implementation; every value is + # exactly representable in bfloat16 (and in float32). + expected = np.array( + [0.81640625, 0.11328125, 0.65234375, 0.76171875], dtype=np.float32 + ) + assert_allclose(got.astype(np.float32), expected, rtol=0, atol=0) + + def test_onnxt_runtime_random_uniform_philox_element_independence(self): + # Intent: Philox is counter-based, so element i depends only on + # (seed_int64, i) — never on how many elements are generated. The + # row-major values of a smaller tensor must therefore be a prefix of + # any larger tensor with the same seed, across counter-block + # boundaries (4 words per block; 26 elements span 7 blocks, the last + # one partially). + def run_philox(shape, seed_int64): + node1 = make_node( + "RandomUniform", + [], + ["Y"], + seed_int64=seed_int64, + shape=shape, + generator="philox4x32_10", + ) + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + return ReferenceEvaluator(onnx_model).run(None, {})[0].ravel() + + small = run_philox([3], 99) + medium = run_philox([2, 3], 99) + large = run_philox([13, 2], 99) + assert_allclose(medium[:3], small, rtol=0, atol=0) + assert_allclose(large[:6], medium, rtol=0, atol=0) + # A different seed keys every block differently. + other_seed = run_philox([13, 2], 100) + self.assertFalse(np.array_equal(large, other_seed)) + + def test_onnxt_runtime_random_uniform_philox_offset_streaming(self): + # Intent: streaming — feeding a different offset per run (e.g. a step + # counter maintained by the host) must yield a fresh, disjoint stream + # per run, while each run individually stays deterministic and + # replayable; omitting the offset input must equal offset = 0. + offset_in = make_tensor_value_info("offset", TensorProto.INT64, []) + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) + node1 = make_node( + "RandomUniform", + ["offset"], + ["Y"], + seed_int64=42, + shape=[2, 3], + generator="philox4x32_10", + ) + graph = make_graph([node1], "g", [offset_in], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + sess = ReferenceEvaluator(onnx_model) + + # The host advances the offset between runs (step counter). + y0 = sess.run(None, {"offset": np.array(0, dtype=np.int64)})[0] + y1 = sess.run(None, {"offset": np.array(1, dtype=np.int64)})[0] + # Different offsets select disjoint streams: no value reappears. + self.assertFalse(np.intersect1d(y0, y1).size) + # Each run is individually replayable. + y0_again = sess.run(None, {"offset": np.array(0, dtype=np.int64)})[0] + assert_allclose(y0_again, y0, rtol=0, atol=0) + + # A model without the offset input behaves like offset = 0. + node2 = make_node( + "RandomUniform", + [], + ["Y"], + seed_int64=42, + shape=[2, 3], + generator="philox4x32_10", + ) + graph2 = make_graph([node2], "g", [], [Y]) + model2 = make_model(graph2) + check_model(model2) + y_default = ReferenceEvaluator(model2).run(None, {})[0] + assert_allclose(y_default, y0, rtol=0, atol=0) + + def test_onnxt_runtime_random_uniform_philox_scalar_shape(self): + # An empty shape attribute produces a scalar output, which is fully + # specified for the deterministic generator: the single element uses + # word 0 of block 0. + Y = make_tensor_value_info("Y", TensorProto.FLOAT, []) + node1 = make_node( + "RandomUniform", [], ["Y"], seed_int64=42, generator="philox4x32_10" + ) + node1.attribute.append( + onnx.helper.make_attribute("shape", [], attr_type=AttributeProto.INTS) + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + check_model(onnx_model) + got = ReferenceEvaluator(onnx_model).run(None, {})[0] + self.assertEqual(got.shape, ()) + self.assertEqual(got.dtype, np.float32) + assert_allclose(got, np.float32(0.61295986), rtol=0, atol=0) + + def test_onnxt_runtime_random_uniform_philox_no_seed_raises(self): + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) + node1 = make_node( + "RandomUniform", [], ["Y"], shape=[2, 3], generator="philox4x32_10" + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + sess = ReferenceEvaluator(onnx_model) + with self.assertRaises(ValueError): + sess.run(None, {}) + + def test_onnxt_runtime_random_uniform_philox_float_seed_raises(self): + # The float seed attribute is forbidden alongside a deterministic + # generator; the reference must enforce this like shape inference. + Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) + node1 = make_node( + "RandomUniform", + [], + ["Y"], + shape=[2, 3], + seed=0.0, + seed_int64=0, + generator="philox4x32_10", + ) + graph = make_graph([node1], "g", [], [Y]) + onnx_model = make_model(graph) + sess = ReferenceEvaluator(onnx_model) + with self.assertRaises(ValueError): + sess.run(None, {}) + + def test_philox4x32_10_known_answer_vectors(self): + # Known-answer vectors from the Random123 distribution + # (tests/kat_vectors, "philox4x32 10" entries): counter and key words + # followed by the expected four output words. + from onnx.reference.ops._op_common_random import ( # noqa: PLC0415 + _Philox4x32, + ) + + kat_vectors = [ + ( + (0x00000000, 0x00000000, 0x00000000, 0x00000000), + (0x00000000, 0x00000000), + (0x6627E8D5, 0xE169C58D, 0xBC57AC4C, 0x9B00DBD8), + ), + ( + (0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF), + (0xFFFFFFFF, 0xFFFFFFFF), + (0x408F276D, 0x41C83B0E, 0xA20BC7C6, 0x6D5451FD), + ), + ( + (0x243F6A88, 0x85A308D3, 0x13198A2E, 0x03707344), + (0xA4093822, 0x299F31D0), + (0xD16CFE09, 0x94FDCCEB, 0x5001E420, 0x24126EA1), + ), + ] + for counter, key, expected in kat_vectors: + out = _Philox4x32.philox4x32_10( + np.uint32([counter[0]]), + np.uint32([counter[1]]), + np.uint32([counter[2]]), + np.uint32([counter[3]]), + key[0], + key[1], + ) + self.assertEqual(tuple(int(w[0]) for w in out), expected) + def test_onnxt_runtime_random_uniform_like(self): X = make_tensor_value_info("X", TensorProto.FLOAT, [None]) Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None]) diff --git a/onnx/test/schema_test.py b/onnx/test/schema_test.py index f135f56fa9b..cae87e7561a 100644 --- a/onnx/test/schema_test.py +++ b/onnx/test/schema_test.py @@ -78,6 +78,25 @@ def allowed(schema): self.assertTrue(celu28.has_function) self.assertEqual(allowed(defs.get_schema("Celu", 12)), {"tensor(float)"}) + def test_randomuniform_generator_attribute(self) -> None: + schema28 = defs.get_schema("RandomUniform", 28) + self.assertIn("generator", schema28.attributes) + generator = schema28.attributes["generator"] + self.assertEqual(generator.type, defs.OpSchema.AttrType.STRING) + self.assertEqual(generator.default_value.s, b"unspecified") + self.assertFalse(generator.required) + # The 64-bit seed for deterministic generators is a separate INT + # attribute; the legacy float seed only applies to "unspecified". + seed_int64 = schema28.attributes["seed_int64"] + self.assertEqual(seed_int64.type, defs.OpSchema.AttrType.INT) + self.assertFalse(seed_int64.required) + # The operator stays non-deterministic at the schema level: with the + # "unspecified" generator the output is still implementation-defined. + self.assertTrue(schema28.non_deterministic) + schema22 = defs.get_schema("RandomUniform", 22) + self.assertNotIn("generator", schema22.attributes) + self.assertNotIn("seed_int64", schema22.attributes) + def test_range_supported_types(self) -> None: """Test Range operator supports all expected numeric types.""" range_schema = defs.get_schema("Range") diff --git a/onnx/test/shape_inference_test.py b/onnx/test/shape_inference_test.py index a2a500d8425..a9a4281abff 100644 --- a/onnx/test/shape_inference_test.py +++ b/onnx/test/shape_inference_test.py @@ -4404,6 +4404,113 @@ def test_random_normal(self) -> None: graph, [make_tensor_value_info("out", TensorProto.DOUBLE, (3, 4, 5))] ) + def test_random_uniform_philox(self) -> None: + graph = self._make_graph( + [], + [ + make_node( + "RandomUniform", + [], + ["out"], + dtype=TensorProto.DOUBLE, + shape=(3, 4), + seed_int64=42, + generator="philox4x32_10", + ) + ], + [], + ) + self._assert_inferred( + graph, [make_tensor_value_info("out", TensorProto.DOUBLE, (3, 4))] + ) + + def test_random_uniform_philox_with_float_seed_fails(self) -> None: + graph = self._make_graph( + [], + [ + make_node( + "RandomUniform", + [], + ["out"], + shape=(3, 4), + seed=42.0, + seed_int64=42, + generator="philox4x32_10", + ) + ], + [], + ) + self.assertRaises(onnx.shape_inference.InferenceError, self._inferred, graph) + + def test_random_uniform_offset(self) -> None: + graph = self._make_graph( + [("offset", TensorProto.INT64, ())], + [ + make_node( + "RandomUniform", + ["offset"], + ["out"], + shape=(3, 4), + seed_int64=0, + generator="philox4x32_10", + ) + ], + [], + ) + self._assert_inferred( + graph, [make_tensor_value_info("out", TensorProto.FLOAT, (3, 4))] + ) + + def test_random_uniform_offset_non_scalar_fails(self) -> None: + graph = self._make_graph( + [("offset", TensorProto.INT64, (2, 3))], + [ + make_node( + "RandomUniform", + ["offset"], + ["out"], + shape=(3, 4), + seed_int64=0, + generator="philox4x32_10", + ) + ], + [], + ) + self.assertRaises(onnx.shape_inference.InferenceError, self._inferred, graph) + + def test_random_uniform_unknown_generator_fails(self) -> None: + graph = self._make_graph( + [], + [ + make_node( + "RandomUniform", + [], + ["out"], + shape=(3, 4), + seed=0.0, + generator="xorshift", + ) + ], + [], + ) + self.assertRaises(onnx.shape_inference.InferenceError, self._inferred, graph) + + def test_random_uniform_philox_without_seed_fails(self) -> None: + graph = self._make_graph( + [], + [ + make_node( + "RandomUniform", + [], + ["out"], + shape=(3, 4), + generator="philox4x32_10", + ) + ], + [], + ) + self.assertRaises(onnx.shape_inference.InferenceError, self._inferred, graph) + def test_random_normal_like(self) -> None: graph = self._make_graph( [("X", TensorProto.FLOAT, (2, 3, 4))], diff --git a/onnx/test/test_backend_reference.py b/onnx/test/test_backend_reference.py index cc6b65461ad..d89dda28818 100644 --- a/onnx/test/test_backend_reference.py +++ b/onnx/test/test_backend_reference.py @@ -168,6 +168,8 @@ def run_node(cls, node, inputs, device=None, outputs_info=None, **kwargs): # tolerance for the expanded case applies only on NumPy >= 2.0. backend_test.exclude(r"test_celu_bfloat16_cpu") backend_test.exclude(r"test_celu_bfloat16_expanded_cpu") + # bfloat16 (ml_dtypes) requires NumPy >= 2.0. + backend_test.exclude(r"test_randomuniform_philox_bfloat16_cpu") # The documentation does not explicitly say that is_causal=1 and attn_mask is not None # is not allowed. The expansion (based on the function definition in ONNX) diff --git a/onnx/test/version_converter_test.py b/onnx/test/version_converter_test.py index 2d667a91aa2..334bcd183d0 100644 --- a/onnx/test/version_converter_test.py +++ b/onnx/test/version_converter_test.py @@ -2900,3 +2900,108 @@ def test_celu_float_27_28_and_28_27(self) -> None: ) def test_celu_28_27_unsupported_type_fails(self, _: str, dtype: int) -> None: self.assertRaises(RuntimeError, lambda: self._celu_converted(dtype, 28, 27)) + + def _randomuniform_converted(self, src: int, dst: int, **attrs) -> ModelProto: + node = helper.make_node("RandomUniform", [], ["Y"], shape=[2, 3], **attrs) + graph = helper.make_graph( + [node], + "randomuniform", + [], + [helper.make_tensor_value_info("Y", TensorProto.FLOAT, [2, 3])], + ) + return self._converted(graph, helper.make_operatorsetid("", src), dst) + + # RandomUniform 27 -> 28: CompatibleAdapter (generator attribute has a default) + def test_randomuniform_27_28(self) -> None: + converted = self._randomuniform_converted(27, 28, seed=0.0) + assert converted.opset_import[0].version == 28 + + # RandomUniform 28 -> 27: generator="unspecified" matches the old + # implementation-defined behavior, so the attribute is dropped on downgrade + def test_randomuniform_28_27_unspecified_generator_removed(self) -> None: + converted = self._randomuniform_converted(28, 27, generator="unspecified") + assert converted.opset_import[0].version == 27 + node = next(n for n in converted.graph.node if n.op_type == "RandomUniform") + assert not any(a.name == "generator" for a in node.attribute) + + # RandomUniform 28 -> 27: a deterministic generator cannot be expressed in + # older opsets and must be rejected + def test_randomuniform_28_27_philox_fails(self) -> None: + self.assertRaises( + RuntimeError, + lambda: self._randomuniform_converted( + 28, 27, generator="philox4x32_10", seed_int64=42 + ), + ) + + # RandomUniform 28 -> 27: the seed_int64 attribute cannot be expressed in + # older opsets and must be rejected + def test_randomuniform_28_27_seed_int64_fails(self) -> None: + self.assertRaises( + RuntimeError, + lambda: self._randomuniform_converted(28, 27, seed_int64=5), + ) + + # RandomUniform 28 -> 27: an omitted optional offset input, spelled as an + # empty string, must not block the downgrade (the placeholder is dropped) + def test_randomuniform_28_27_empty_offset_placeholder(self) -> None: + node = helper.make_node("RandomUniform", [""], ["Y"], shape=[2, 3], seed=1.0) + graph = helper.make_graph( + [node], + "randomuniform_empty_offset", + [], + [helper.make_tensor_value_info("Y", TensorProto.FLOAT, [2, 3])], + ) + converted = self._converted(graph, helper.make_operatorsetid("", 28), 27) + assert converted.opset_import[0].version == 27 + ru = next(n for n in converted.graph.node if n.op_type == "RandomUniform") + assert not [i for i in ru.input if i] + + # RandomUniform 28 -> 27: the offset input cannot be expressed in older + # opsets and must be rejected + def test_randomuniform_28_27_offset_fails(self) -> None: + node = helper.make_node( + "RandomUniform", ["offset"], ["Y"], shape=[2, 3], seed=1.0 + ) + graph = helper.make_graph( + [node], + "randomuniform_offset", + [helper.make_tensor_value_info("offset", TensorProto.INT64, [])], + [helper.make_tensor_value_info("Y", TensorProto.FLOAT, [2, 3])], + ) + self.assertRaises( + RuntimeError, + lambda: self._converted(graph, helper.make_operatorsetid("", 28), 27), + ) + + def _randomuniform_offset_initializer(self, offset_value: int) -> ModelProto: + node = helper.make_node( + "RandomUniform", ["offset"], ["Y"], shape=[2, 3], seed=1.0 + ) + graph = helper.make_graph( + [node], + "randomuniform_offset_initializer", + [], + [helper.make_tensor_value_info("Y", TensorProto.FLOAT, [2, 3])], + initializer=[ + helper.make_tensor("offset", TensorProto.INT64, [], [offset_value]) + ], + ) + return self._converted(graph, helper.make_operatorsetid("", 28), 27) + + # RandomUniform 28 -> 27: a constant offset of 0 — the documented pattern + # for storing the stream position in the model — matches the default and + # is dropped together with its initializer + def test_randomuniform_28_27_constant_zero_offset_removed(self) -> None: + converted = self._randomuniform_offset_initializer(0) + assert converted.opset_import[0].version == 27 + ru = next(n for n in converted.graph.node if n.op_type == "RandomUniform") + assert not [i for i in ru.input if i] + assert not [i for i in converted.graph.initializer if i.name == "offset"] + + # RandomUniform 28 -> 27: a non-zero constant offset selects a stream that + # older opsets cannot express and must be rejected + def test_randomuniform_28_27_constant_nonzero_offset_fails(self) -> None: + self.assertRaises( + RuntimeError, lambda: self._randomuniform_offset_initializer(5) + ) diff --git a/onnx/version_converter/adapters/CMakeLists.txt b/onnx/version_converter/adapters/CMakeLists.txt index d4afa967b36..f8ae7f379ec 100644 --- a/onnx/version_converter/adapters/CMakeLists.txt +++ b/onnx/version_converter/adapters/CMakeLists.txt @@ -24,6 +24,7 @@ target_sources(onnx PRIVATE no_previous_version.h pad_10_11.h q_dq_21_20.h + random_generator_28_27.h remove_consumed_inputs.h reshape_4_5.h reshape_5_4.h diff --git a/onnx/version_converter/adapters/random_generator_28_27.h b/onnx/version_converter/adapters/random_generator_28_27.h new file mode 100644 index 00000000000..54ca4ead2c5 --- /dev/null +++ b/onnx/version_converter/adapters/random_generator_28_27.h @@ -0,0 +1,112 @@ +// Copyright (c) ONNX Project Contributors +// +// SPDX-License-Identifier: Apache-2.0 + +// Adapter for the random-generator operators in default domain from version +// 28 to 27 (currently RandomUniform; intended for the other random operators +// when they adopt the deterministic generator mechanism). + +#pragma once + +#include +#include +#include +#include + +#include "onnx/version_converter/adapters/adapter.h" +#include "onnx/version_converter/helper.h" + +namespace ONNX_NAMESPACE { +namespace version_conversion { + +class RandomGenerator_28_27 final : public Adapter { + public: + RandomGenerator_28_27(std::string op_name, size_t num_legacy_inputs) + : Adapter(std::move(op_name), OpSetID(28), OpSetID(27)), num_legacy_inputs_(num_legacy_inputs) {} + + Node* adapt(std::shared_ptr graph, Node* node) const override { + // The optional offset input (the first input after the operator's legacy + // inputs) does not exist before version 28. It can be removed without + // changing semantics when it is omitted — possibly spelled as an empty + // string, which the proto importer materializes as a kUndefined + // placeholder — or when it is a constant 0, the documented pattern for + // storing the stream position in the model. Any other offset selects a + // stream that older versions cannot express. + if (node->inputs().size() > num_legacy_inputs_) { + ONNX_ASSERTM( + node->inputs().size() == num_legacy_inputs_ + 1 && IsRemovableOffset(graph, node), + "Operator '", + name(), + "' with an 'offset' input is not supported in Opset Version ", + static_cast(target_version().version()), + " (only an omitted offset or a constant offset of 0 can be removed)."); + RemoveOffsetInput(graph, node); + } + // seed_int64 does not exist before version 28. + ONNX_ASSERTM( + !node->hasAttribute(Symbol("seed_int64")), + "Attribute 'seed_int64' of operator '", + name(), + "' is not supported in Opset Version ", + static_cast(target_version().version()), + "."); + const Symbol generator("generator"); + if (node->hasAttribute(generator)) { + // "unspecified" matches the implementation-defined behavior of the + // pre-28 operators, so the attribute can simply be dropped. Any other + // generator selects fully specified deterministic output, which older + // versions cannot express. + ONNX_ASSERTM( + node->s(generator) == "unspecified", + "Attribute 'generator' of operator '", + name(), + "' must be 'unspecified' in Opset Version ", + static_cast(target_version().version()), + "."); + node->removeAttribute(generator); + } + return node; + } + + private: + size_t num_legacy_inputs_; + + bool IsRemovableOffset(const std::shared_ptr& graph, Node* node) const { + const Value* offset_val = node->inputs()[num_legacy_inputs_]; + const Node* offset_node = offset_val->node(); + if (offset_node->kind() == kUndefined) { + return true; + } + if (offset_node->kind() == kConstant) { + const std::vector values = ReadInt64Tensor(offset_node->t(kvalue)); + return values.size() == 1 && values[0] == 0; + } + if (graph->is_constant_initializer(offset_val)) { + for (const auto& initializer : graph->initializers()) { + if (initializer.name() == offset_val->uniqueName()) { + const std::vector values = ReadInt64Tensor(initializer); + return values.size() == 1 && values[0] == 0; + } + } + } + return false; + } + + void RemoveOffsetInput(const std::shared_ptr& graph, Node* node) const { + Value* offset_val = node->inputs()[num_legacy_inputs_]; + Node* offset_node = offset_val->node(); + const std::string initializer_name = offset_val->uniqueName(); + const bool is_initializer = graph->is_constant_initializer(offset_val); + node->removeInput(num_legacy_inputs_); + if (offset_val->uses().empty()) { + if (is_initializer) { + graph->eraseInitializer(initializer_name); + } else if (offset_node->kind() == kConstant) { + offset_node->destroy(); + } + } + } +}; + +} // namespace version_conversion +} // namespace ONNX_NAMESPACE diff --git a/onnx/version_converter/convert.h b/onnx/version_converter/convert.h index 6ad1e7d1d6c..40726eb34d6 100644 --- a/onnx/version_converter/convert.h +++ b/onnx/version_converter/convert.h @@ -38,6 +38,7 @@ #include "onnx/version_converter/adapters/no_previous_version.h" #include "onnx/version_converter/adapters/pad_10_11.h" #include "onnx/version_converter/adapters/q_dq_21_20.h" +#include "onnx/version_converter/adapters/random_generator_28_27.h" #include "onnx/version_converter/adapters/range_27_26.h" #include "onnx/version_converter/adapters/reshape_4_5.h" #include "onnx/version_converter/adapters/reshape_5_4.h" @@ -981,12 +982,16 @@ class DefaultVersionConverter : public BaseVersionConverter { /******** 27 -> 28 ********/ registerAdapter(std::make_unique("Celu", OpSetID(27), OpSetID(28))); + registerAdapter(std::make_unique("RandomUniform", OpSetID(27), OpSetID(28))); /******** 28 -> 27 ********/ // Celu v28 widened T to all_float_types_ir4(); Celu v12 (opset 27) supports only FLOAT. const std::vector celu_28_unallowed_types = { TensorProto_DataType_FLOAT16, TensorProto_DataType_BFLOAT16, TensorProto_DataType_DOUBLE}; registerAdapter(std::make_unique("Celu", OpSetID(28), OpSetID(27), celu_28_unallowed_types)); + // RandomUniform v28 added the generator/seed_int64 attributes and the offset input; + // only generator="unspecified" without seed_int64 and offset can be downgraded. + registerAdapter(std::make_unique("RandomUniform", 0)); } ModelProto convert_version(const ModelProto& mp_in, const OpSetID& initial_version, const OpSetID& target_version)