fix: make DynamicValue#toTyped stack safe using trampolining (#422)#1031
fix: make DynamicValue#toTyped stack safe using trampolining (#422)#1031
Conversation
There was a problem hiding this comment.
Pull request overview
Updates DynamicValue#toTyped decoding to be stack-safe for deeply nested values by replacing direct recursion with trampolined evaluation (scala.util.control.TailCalls), and re-enables the previously ignored stack-safety test from #421/#422.
Changes:
- Replaced recursive
toTypedValueLazyErrorlogic with a trampoline-based implementation (toTypedValueTrampoline). - Updated
decodeStructureto use a trampolined loop (decodeStructureTrampoline) to avoid recursive calls. - Un-ignored and adjusted the
"toTyped is stack safe"property test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| zio-schema/shared/src/main/scala/zio/schema/DynamicValue.scala | Migrates toTyped/record decoding recursion to TailCalls trampolining for stack safety. |
| tests/shared/src/test/scala/zio/schema/DynamicValueSpec.scala | Re-enables the stack-safety test for toTyped with deeper generated JSON. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| case head :: tail => | ||
| tailcall(toTypedValueTrampoline(head, schema.elementSchema).asInstanceOf[TailRec[Either[DecodeError, t]]]).flatMap { | ||
| case Left(err) => done(Left(err)) |
There was a problem hiding this comment.
The new trampoline-based collection loops rely on asInstanceOf[TailRec[Either[DecodeError, t]]] casts. These casts are unsafe and can mask real type issues at compile time; it should be possible to avoid them by introducing a properly-typed helper (e.g., a local def decodeElem(e: DynamicValue): TailRec[Either[DecodeError, t]]) or by adding an explicit type parameter when calling toTypedValueTrampoline so the compiler can infer A = t without casting.
There was a problem hiding this comment.
Fixed — replaced the asInstanceOf cast with an explicit type parameter: toTypedValueTrampoline[t](head, schema.elementSchema). The compiler infers the correct existential type when given the explicit parameter.
| case head :: tail => | ||
| tailcall(toTypedValueTrampoline(head, schema.elementSchema).asInstanceOf[TailRec[Either[DecodeError, t]]]).flatMap { | ||
| case Left(err) => done(Left(err)) | ||
| case Right(typed) => loopSet(tail, acc + typed) |
There was a problem hiding this comment.
This asInstanceOf[TailRec[Either[DecodeError, t]]] cast in the Set decoding loop is unsafe. Consider refactoring to a typed helper or explicitly parameterizing toTypedValueTrampoline so the element schema type is preserved without casts.
There was a problem hiding this comment.
Fixed — same approach as above: explicit type parameter [t] on toTypedValueTrampoline.
| case (dk, dv) :: tail => | ||
| tailcall(toTypedValueTrampoline(dk, schema.keySchema).asInstanceOf[TailRec[Either[DecodeError, k]]]).flatMap { | ||
| case Left(err) => done(Left(err)) | ||
| case Right(key) => | ||
| tailcall(toTypedValueTrampoline(dv, schema.valueSchema).asInstanceOf[TailRec[Either[DecodeError, v]]]).flatMap { | ||
| case Left(err) => done(Left(err)) |
There was a problem hiding this comment.
This dictionary decoding loop uses asInstanceOf[TailRec[Either[DecodeError, k]]] / asInstanceOf[TailRec[Either[DecodeError, v]]] casts. Since these casts bypass the type checker, it would be safer/clearer to introduce typed helper methods or explicit type parameters to toTypedValueTrampoline to avoid asInstanceOf.
There was a problem hiding this comment.
Fixed — both key and value casts replaced with explicit type parameters: toTypedValueTrampoline[k](dk, schema.keySchema) and toTypedValueTrampoline[v](dv, schema.valueSchema).
| val dyn = DynamicValue.fromSchemaAndValue(Json.schema, json) | ||
| val result = dyn.toTypedValue(Json.schema) | ||
| if (result != Right(json)) | ||
| throw new AssertionError("toTyped round-trip failed") | ||
| assertCompletes |
There was a problem hiding this comment.
The test now throws an AssertionError manually and then returns assertCompletes. Using ZIO Test assertions (e.g., assert(result)(equalTo(Right(json))) / assertTrue(result == Right(json))) will produce better failure reporting and avoids relying on exceptions for control flow.
There was a problem hiding this comment.
Intentionally kept as-is. Using assertTrue(result == Right(json)) causes ZIO Test's PrettyPrint to recursively pretty-print the deeply nested JSON structure on failure, which itself triggers a StackOverflowError (the PrettyPrint is not stack-safe for deeply recursive data types). The throw AssertionError + assertCompletes pattern is a workaround for this ZIO Test limitation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| val constructed = s.construct(Chunk.fromIterable(m.values))(Unsafe.unsafe) | ||
| done(constructed.left.map(err => DecodeError.MalformedField(s, err))) |
There was a problem hiding this comment.
Schema.Record construction uses Chunk.fromIterable(m.values), which depends on the iteration order of m (ultimately the insertion order of values) matching s.fields order. Schema.Record.construct indexes the chunk positionally, so out-of-order ListMap keys can silently swap same-typed fields or cause spurious "invalid type in values" failures. Build the Chunk[Any] in s.fields order (by field name) instead of relying on m.values ordering, or change decodeStructureTrampoline to traverse structure in order and accumulate accordingly.
| val constructed = s.construct(Chunk.fromIterable(m.values))(Unsafe.unsafe) | |
| done(constructed.left.map(err => DecodeError.MalformedField(s, err))) | |
| // Build the chunk in schema field order to avoid relying on ListMap insertion order. | |
| val fieldValuesEither: Either[DecodeError, Chunk[Any]] = | |
| s.fields.foldLeft[Either[DecodeError, Chunk[Any]]](Right(Chunk.empty)) { (accEither, field) => | |
| accEither match { | |
| case Left(e) => Left(e) | |
| case Right(accChunk) => | |
| m.get(field.name) match { | |
| case Some(v) => Right(accChunk :+ v) | |
| case None => Left(DecodeError.IncompatibleShape(values, s.fields)) | |
| } | |
| } | |
| } | |
| fieldValuesEither match { | |
| case Left(err) => | |
| done(Left(err)) | |
| case Right(fieldValues) => | |
| val constructed = s.construct(fieldValues)(Unsafe.unsafe) | |
| done(constructed.left.map(err => DecodeError.MalformedField(s, err))) | |
| } |
| val keys = values.keySet.toList | ||
| def loop(remaining: List[String], acc: ListMap[String, Any]): TailRec[Either[DecodeError, ListMap[String, Any]]] = | ||
| remaining match { | ||
| case Nil => done(Right(acc)) | ||
| case key :: rest => | ||
| (structure.find(_.name == key), values.get(key)) match { | ||
| case (Some(field), Some(value)) => | ||
| tailcall(toTypedValueTrampoline(value, field.schema)).flatMap { | ||
| case Left(err) => done(Left(err)) | ||
| case Right(typed) => loop(rest, acc + (key -> typed)) | ||
| } |
There was a problem hiding this comment.
decodeStructureTrampoline does a linear structure.find(_.name == key) for every key, yielding O(n²) behavior for wide records. Since this method is on the decoding hot path, consider pre-indexing structure into a Map[String, Field[_, _]] once (or iterating structure and looking up values) to make lookups O(1) and reduce allocations.
… trampolined toTyped Use explicit type parameters [t], [k], [v] on toTypedValueTrampoline calls in Sequence, Set, and Dictionary loops to eliminate unsafe asInstanceOf casts. The compiler can infer the correct existential types when given explicit type parameters, making the casts unnecessary.
e5a9062 to
5edffdf
Compare
Summary
Fixes #422
DynamicValue#toTypeduses direct recursion which causesStackOverflowErroron deeply nested structures. This PR makes it stack-safe usingscala.util.control.TailCalls(trampolining).Changes
toTypedValueLazyErrorfrom direct recursion to a trampoline-basedtoTypedValueTrampolinemethoddecodeStructureto use trampoline viadecodeStructureTrampolinetailcall()instead of direct recursion"toTyped is stack safe"test (was@@ TestAspect.ignoresince Use CaseClass0 for representing schema of case objects #421)Test Results