Skip to content

fix: make DynamicValue#toTyped stack safe using trampolining (#422)#1031

Open
987Nabil wants to merge 3 commits intozio:mainfrom
987Nabil:fix/422-dynamic-value-stack-safe
Open

fix: make DynamicValue#toTyped stack safe using trampolining (#422)#1031
987Nabil wants to merge 3 commits intozio:mainfrom
987Nabil:fix/422-dynamic-value-stack-safe

Conversation

@987Nabil
Copy link
Copy Markdown
Contributor

@987Nabil 987Nabil commented Mar 8, 2026

Summary

Fixes #422

DynamicValue#toTyped uses direct recursion which causes StackOverflowError on deeply nested structures. This PR makes it stack-safe using scala.util.control.TailCalls (trampolining).

Changes

  • Converted toTypedValueLazyError from direct recursion to a trampoline-based toTypedValueTrampoline method
  • Converted decodeStructure to use trampoline via decodeStructureTrampoline
  • Every recursive call now uses tailcall() instead of direct recursion
  • Collection processing (Sequence, Set, Dictionary) uses iterative loop helpers within the trampoline
  • Un-ignored the existing "toTyped is stack safe" test (was @@ TestAspect.ignore since Use CaseClass0 for representing schema of case objects #421)
  • No changes to the public API

Test Results

  • 539 tests pass, 0 failures
  • The previously-ignored stack safety test now passes

@987Nabil 987Nabil requested a review from a team as a code owner March 8, 2026 23:00
@987Nabil 987Nabil requested a review from Copilot March 11, 2026 07:44
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates DynamicValue#toTyped decoding to be stack-safe for deeply nested values by replacing direct recursion with trampolined evaluation (scala.util.control.TailCalls), and re-enables the previously ignored stack-safety test from #421/#422.

Changes:

  • Replaced recursive toTypedValueLazyError logic with a trampoline-based implementation (toTypedValueTrampoline).
  • Updated decodeStructure to use a trampolined loop (decodeStructureTrampoline) to avoid recursive calls.
  • Un-ignored and adjusted the "toTyped is stack safe" property test.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
zio-schema/shared/src/main/scala/zio/schema/DynamicValue.scala Migrates toTyped/record decoding recursion to TailCalls trampolining for stack safety.
tests/shared/src/test/scala/zio/schema/DynamicValueSpec.scala Re-enables the stack-safety test for toTyped with deeper generated JSON.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +179 to +181
case head :: tail =>
tailcall(toTypedValueTrampoline(head, schema.elementSchema).asInstanceOf[TailRec[Either[DecodeError, t]]]).flatMap {
case Left(err) => done(Left(err))
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new trampoline-based collection loops rely on asInstanceOf[TailRec[Either[DecodeError, t]]] casts. These casts are unsafe and can mask real type issues at compile time; it should be possible to avoid them by introducing a properly-typed helper (e.g., a local def decodeElem(e: DynamicValue): TailRec[Either[DecodeError, t]]) or by adding an explicit type parameter when calling toTypedValueTrampoline so the compiler can infer A = t without casting.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — replaced the asInstanceOf cast with an explicit type parameter: toTypedValueTrampoline[t](head, schema.elementSchema). The compiler infers the correct existential type when given the explicit parameter.

Comment on lines +191 to +194
case head :: tail =>
tailcall(toTypedValueTrampoline(head, schema.elementSchema).asInstanceOf[TailRec[Either[DecodeError, t]]]).flatMap {
case Left(err) => done(Left(err))
case Right(typed) => loopSet(tail, acc + typed)
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asInstanceOf[TailRec[Either[DecodeError, t]]] cast in the Set decoding loop is unsafe. Consider refactoring to a typed helper or explicitly parameterizing toTypedValueTrampoline so the element schema type is preserved without casts.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — same approach as above: explicit type parameter [t] on toTypedValueTrampoline.

Comment on lines +214 to +219
case (dk, dv) :: tail =>
tailcall(toTypedValueTrampoline(dk, schema.keySchema).asInstanceOf[TailRec[Either[DecodeError, k]]]).flatMap {
case Left(err) => done(Left(err))
case Right(key) =>
tailcall(toTypedValueTrampoline(dv, schema.valueSchema).asInstanceOf[TailRec[Either[DecodeError, v]]]).flatMap {
case Left(err) => done(Left(err))
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dictionary decoding loop uses asInstanceOf[TailRec[Either[DecodeError, k]]] / asInstanceOf[TailRec[Either[DecodeError, v]]] casts. Since these casts bypass the type checker, it would be safer/clearer to introduce typed helper methods or explicit type parameters to toTypedValueTrampoline to avoid asInstanceOf.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — both key and value casts replaced with explicit type parameters: toTypedValueTrampoline[k](dk, schema.keySchema) and toTypedValueTrampoline[v](dv, schema.valueSchema).

Comment on lines +98 to +102
val dyn = DynamicValue.fromSchemaAndValue(Json.schema, json)
val result = dyn.toTypedValue(Json.schema)
if (result != Right(json))
throw new AssertionError("toTyped round-trip failed")
assertCompletes
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test now throws an AssertionError manually and then returns assertCompletes. Using ZIO Test assertions (e.g., assert(result)(equalTo(Right(json))) / assertTrue(result == Right(json))) will produce better failure reporting and avoids relying on exceptions for control flow.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentionally kept as-is. Using assertTrue(result == Right(json)) causes ZIO Test's PrettyPrint to recursively pretty-print the deeply nested JSON structure on failure, which itself triggers a StackOverflowError (the PrettyPrint is not stack-safe for deeply recursive data types). The throw AssertionError + assertCompletes pattern is a workaround for this ZIO Test limitation.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +146 to +147
val constructed = s.construct(Chunk.fromIterable(m.values))(Unsafe.unsafe)
done(constructed.left.map(err => DecodeError.MalformedField(s, err)))
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema.Record construction uses Chunk.fromIterable(m.values), which depends on the iteration order of m (ultimately the insertion order of values) matching s.fields order. Schema.Record.construct indexes the chunk positionally, so out-of-order ListMap keys can silently swap same-typed fields or cause spurious "invalid type in values" failures. Build the Chunk[Any] in s.fields order (by field name) instead of relying on m.values ordering, or change decodeStructureTrampoline to traverse structure in order and accumulate accordingly.

Suggested change
val constructed = s.construct(Chunk.fromIterable(m.values))(Unsafe.unsafe)
done(constructed.left.map(err => DecodeError.MalformedField(s, err)))
// Build the chunk in schema field order to avoid relying on ListMap insertion order.
val fieldValuesEither: Either[DecodeError, Chunk[Any]] =
s.fields.foldLeft[Either[DecodeError, Chunk[Any]]](Right(Chunk.empty)) { (accEither, field) =>
accEither match {
case Left(e) => Left(e)
case Right(accChunk) =>
m.get(field.name) match {
case Some(v) => Right(accChunk :+ v)
case None => Left(DecodeError.IncompatibleShape(values, s.fields))
}
}
}
fieldValuesEither match {
case Left(err) =>
done(Left(err))
case Right(fieldValues) =>
val constructed = s.construct(fieldValues)(Unsafe.unsafe)
done(constructed.left.map(err => DecodeError.MalformedField(s, err)))
}

Copilot uses AI. Check for mistakes.
Comment on lines +115 to +125
val keys = values.keySet.toList
def loop(remaining: List[String], acc: ListMap[String, Any]): TailRec[Either[DecodeError, ListMap[String, Any]]] =
remaining match {
case Nil => done(Right(acc))
case key :: rest =>
(structure.find(_.name == key), values.get(key)) match {
case (Some(field), Some(value)) =>
tailcall(toTypedValueTrampoline(value, field.schema)).flatMap {
case Left(err) => done(Left(err))
case Right(typed) => loop(rest, acc + (key -> typed))
}
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

decodeStructureTrampoline does a linear structure.find(_.name == key) for every key, yielding O(n²) behavior for wide records. Since this method is on the decoding hot path, consider pre-indexing structure into a Map[String, Field[_, _]] once (or iterating structure and looking up values) to make lookups O(1) and reduce allocations.

Copilot uses AI. Check for mistakes.
… trampolined toTyped

Use explicit type parameters [t], [k], [v] on toTypedValueTrampoline calls
in Sequence, Set, and Dictionary loops to eliminate unsafe asInstanceOf casts.
The compiler can infer the correct existential types when given explicit
type parameters, making the casts unnecessary.
@987Nabil 987Nabil force-pushed the fix/422-dynamic-value-stack-safe branch from e5a9062 to 5edffdf Compare March 11, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DynamicValue#toTyped is not stack safe

2 participants