fix(jsonata): bound maxDepth by expression nesting, not input size - #103
Conversation
Frame.setRuntimeBounds installs a Timebox whose entry and exit callbacks both skip accounting when the frame carries isParallelCall. The two sides do not observe the same flag for a given evaluate() call, so the counter gains one level per input item and never unwinds: maxDepth ended up capping how many records could be processed rather than how deeply the expression nests. A 1000-item batch failed on the default maxDepth=1000, reporting non-terminating recursion for expressions containing none. Replace it with EvaluationBounds, which counts entry and exit unconditionally — safe because the Java port evaluates path steps sequentially. Depth is now constant in input size (4 levels for a $map projection at any batch size), and runaway recursion and timeouts are still caught, with actionable messages. The companion defect in #102 — $map with a lambda being quadratic — is upstream in dashjoin/jsonata-java and unfixable from here; the plugin doc now points at the equivalent path projection, which is linear. Refs #102 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
📦 Artifacts
🛡 TrivyVulnerability in:
|
| Vulnerability | Severity | Package | Installed Version | Fixed Version |
|---|---|---|---|---|
| GHSA-r7wm-3cxj-wff9 | HIGH | com.fasterxml.jackson.core:jackson-core | 2.21.1 | 2.18.8, 2.21.4, 2.22.1 |
| GHSA-r7wm-3cxj-wff9 | HIGH | com.fasterxml.jackson.core:jackson-core | 2.21.1 | 2.18.8, 2.21.4, 2.22.1 |
| GHSA-r7wm-3cxj-wff9 | HIGH | com.fasterxml.jackson.core:jackson-core | 2.21.1 | 2.18.8, 2.21.4, 2.22.1 |
| CVE-2026-54512 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 3.1.4, 2.21.4 |
| CVE-2026-54512 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 3.1.4, 2.21.4 |
| CVE-2026-54512 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 3.1.4, 2.21.4 |
| CVE-2026-54513 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54513 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54513 | HIGH | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54514 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54514 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54514 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4, 3.1.4 |
| CVE-2026-54515 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 3.1.4, 2.18.9, 2.21.5, 2.22.1 |
| CVE-2026-54515 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 3.1.4, 2.18.9, 2.21.5, 2.22.1 |
| CVE-2026-54515 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 3.1.4, 2.18.9, 2.21.5, 2.22.1 |
| CVE-2026-54516 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54516 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54516 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54517 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54517 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54517 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4, 3.1.4 |
| CVE-2026-54518 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4 |
| CVE-2026-54518 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4 |
| CVE-2026-54518 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.4 |
| CVE-2026-59888 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4 |
| CVE-2026-59888 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4 |
| CVE-2026-59888 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.8, 2.21.4 |
| CVE-2026-59889 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.5, 2.18.9, 2.22.1 |
| CVE-2026-59889 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.5, 2.18.9, 2.22.1 |
| CVE-2026-59889 | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.21.5, 2.18.9, 2.22.1 |
| GHSA-mhm7-754m-9p8w | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.9, 2.21.5 |
| GHSA-mhm7-754m-9p8w | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.9, 2.21.5 |
| GHSA-mhm7-754m-9p8w | MEDIUM | com.fasterxml.jackson.core:jackson-databind | 2.21.1 | 2.18.9, 2.21.5 |
🧪 Java Unit Tests
| Tests | Passed ✅ | Skipped | Failed | Time ⏱ | |
|---|---|---|---|---|---|
| Java Tests Report | 150 ran | 149 ✅ | 1 | 0 ❌ | 41s 559ms |
🔁 Unreleased Commits
2 commits since v1.5.6
| SHA | Title | Author | Date |
|---|---|---|---|
5b01888 |
chore(deps): bump com.gradleup.shadow from 9.4.3 to 9.6.1 (#101) | dependabot[bot] | Jul 30, 26, 1:04:48 PM UTC |
43ec99e |
chore(deps): bump the kestra-gradle-plugins group with 3 updates (#98) | dependabot[bot] | Jul 30, 26, 1:04:59 PM UTC |
Tests report quick summary:success ✅ > tests: 150, success: 149, skipped: 1, failed: 0 unfold for details
|
Isolating the shapes shows the quadratic cost needs both a user-defined lambda in $map and an object constructor consuming its result: the same wrapper around a builtin function is flat (10 ms at 1000 items), and the lambda without the wrapper is too (103 ms). The previous wording blamed $map with a lambda in general, which overstates it. Also corrects the figures to the reported expression measured end to end: 18 s as written against 18 ms rewritten, not 39 s against 10 ms. Refs #102 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jymaire
left a comment
There was a problem hiding this comment.
Kestra Plugin Code Review
(Note: posted as COMMENT, not APPROVE — GitHub blocks self-approval since the authenticated account matches the PR author. Verdict below reflects the actual review outcome.)
Business Requirements — met
Issue #102 lists four checkboxes for "defect 1" (the depth-accounting bug); all four are addressed: EvaluationBounds fixes the accounting, the maxDepth @Schema wording is corrected, the plugin how-to doc documents the path-projection preference, and regression tests assert a fixed expression needs a fixed maxDepth regardless of input size. Defect 2 (the $map+lambda quadratic cost, upstream-only) is explicitly out of scope and the PR is honest about it ("Closes the first half of #102") — no scope creep.
I independently verified the root-cause claim against the actual com.dashjoin:jsonata:0.9.10 source (pulled from the tagged release): Timebox's entry/exit callbacks both gate on Frame.isParallelCall, and that flag is set unconditionally (environment.isParallelCall = idx > 0) inside plain sequential Java for loops for array constructors and object-constructor grouping (ported from JS Promise.all, but executed synchronously in the Java port — no threads, no CompletableFuture, only a ThreadLocal for top-level reentrancy). A shared, mutable Frame reference gets its isParallelCall field overwritten by nested loops between a given call's entry and its own exit, breaking the entry/exit symmetry the original Timebox relied on. EvaluationBounds counting unconditionally is therefore correct given the Java port's actual (synchronous) execution model, matching the tests (100/5000-item parity, maxDepth=12 parity at 10/400 items, timeout still enforced).
Kestra Guidelines — 2 findings (both non-blocking, pre-existing/out of scope)
- 🟡
Transform.java:107(unchanged by this PR) —runContext.render(getMaxDepth()).as(Integer.class).orElseThrow()on a property that does declare a default (@Builder.Default private Property<Integer> maxDepth = Property.ofValue(1000)) violates the "render with.orElse(default), never.orElseThrow()" guideline. Pre-existing code, not touched by this diff (only line 111 changed in this file) — flagging for a follow-up, not blocking this PR. - 🟢 New test method bodies (
TransformItemsTest,TransformValueTest) use explicit local types (RunContext runContext = ...,List<Map<String,Object>> batch = ...) instead ofvar, per the "usevarfor all local variables" rule. Consistent with the rest of both files, which already use explicit types throughout — not a regression introduced by this PR, just noting for a broader style pass.
Everything else checked clean: EvaluationBounds is correctly scoped package-private (no Lombok annotations needed — it's a plain internal helper, not a Task/Trigger/Output); the class-level Javadoc explains the non-obvious why (the Timebox asymmetry) without narration; the @Schema description uses a """ text block; the plugin how-to doc was updated to reflect the corrected maxDepth semantics and the $map+lambda gotcha, in line with "Existing plugin" doc-update expectations; the com.dashjoin:jsonata dependency is pinned to an exact version (0.9.10), not a range.
Security (OWASP Top 10:2025 + KPS) — no issues found
No secrets, no new external calls, no new dependencies, no injection surface. The new JException messages only interpolate maxDepth/timeoutMillis integers — no expression content or record data is echoed back, so no data leakage via error messages (A10). One design-level (not OWASP-category) note worth surfacing: the fix depends on an internal execution-model detail of a pinned third-party library version (synchronous, non-parallel evaluation) rather than a documented public contract. That's a reasonable trade-off given the version pin and the issue's stated plan to file both defects upstream, but if com.dashjoin:jsonata is ever bumped and changes its execution model (e.g., adopts real async/parallel evaluation to match its JS origin), EvaluationBounds's unconditional counting could silently regress the same way Timebox did. Worth a short note in the Javadoc or a follow-up ticket to re-verify this assumption on any future version bump — non-blocking.
Performance — no issues found
EvaluationBounds.enter() calls System.currentTimeMillis() on every evaluate-entry event unconditionally, slightly more often than Timebox did in the buggy state (which skipped many checks via the broken isParallelCall gate) — but currentTimeMillis() is a cheap intrinsic and this is inherent to the depth/timeout bound design, not a new hot-path regression. No unbounded allocation, no new shared mutable state across threads (the single depth field is only ever touched from the dedicated single-thread evalExecutor, and a fresh EvaluationBounds instance is created per evaluateExpression() call, so there's no cross-record leakage even if a mid-evaluation exception leaves depth in an inconsistent state within one call).
Verdict: APPROVE
QA Report — browser-based, Kestra OSSEdition: OSS | Docker tag: Verdict: PASS. The reported defect reproduces on the base commit inside the product and is gone with this PR applied, on the same instance and the same flow. The depth bound and the timeout both still fire. No regression on any of the three Two things reviewers should read below rather than skim: my first-pass reproducer did not actually reproduce the bug (§ Flow 9 correction), and the timeout half of Results summary
No scenario hit the 30-second timeout rule. Flow 9:
|
| Task | Status | Duration |
|---|---|---|
gen_2000 |
SUCCESS | 0.18s |
shape_a_object_wrapping_array_literal |
SUCCESS | 0.04s |
shape_b_object_wrapping_mapped_lambda |
WARNING (failed, allowFailure) |
5.77s |
| Total | WARNING | ~6s |
Gantt — this PR (c8c1706)
| Task | Status | Duration |
|---|---|---|
gen_2000 |
SUCCESS | 0.23s |
shape_a_object_wrapping_array_literal |
SUCCESS | 0.22s |
shape_b_object_wrapping_mapped_lambda |
SUCCESS | 10.60s |
| Total | SUCCESS | 11.1s |
Logs synthesis — base commit, verbatim, exactly the error from #102:
ERROR JSonataException Stack overflow error: Check for non-terminating recursive function.
Consider rewriting as tail-recursive. Depth=1001 max=1000
With the PR applied the same task logs no error and returns processedItemsTotal: 1.
The task gets slower after the fix (5.77s → 10.60s), and that is the expected direction: pre-fix it aborted at roughly the 1000th item, post-fix it evaluates all 2000 — and this shape is the quadratic one from defect 2, which is upstream and explicitly out of scope here.
Note on shape_a: the array-literal shape {"items": [$.{…}]} passed at 2000 items on both JARs. #102's second table predicts it should need maxDepth: 3200 at 1000 items. I could not reproduce leaking for that shape. The likely difference is projection width — #102 measured a 45-field projection, mine has 2 keys. So the in-product reproduction I obtained is specifically the $map + user-defined lambda inside an object constructor shape. #102's broader claim that any per-item nested object constructor leaks is not something this QA confirms; it does not affect the verdict on this PR, but the issue's table may be narrower than written.
Correction: flows 4 and 5 are not valid reproducers
I built these two first, expecting them to be the headline evidence. Both pass on this PR — but I re-ran them against the base commit and they pass there too, so their green status proves nothing about the fix. Reporting them rather than dropping them, since a reviewer skimming the pass column would otherwise over-read them.
The expression I used, {"orders": $.{…, "meta": {…}}}, is an object wrapping a path projection — not a leaking shape. Flow 9 is the corrected test.
| Flow | Task | Base commit | This PR |
|---|---|---|---|
4 qa_large_batch_default_maxdepth |
2000 items, default maxDepth: 1000 |
SUCCESS | SUCCESS (0.50s) |
5 qa_maxdepth_input_size_independence |
25 items, maxDepth: 20 |
SUCCESS | SUCCESS (0.18s) |
| 5 | 2000 items, maxDepth: 20 |
SUCCESS | SUCCESS (0.22s) |
What flow 5 does still show, on the PR build: the same expression runs under maxDepth: 20 at both 25 and 2000 items — consistent with the PR's central claim, just not a discriminating test of it.
Flow 6: qa_maxdepth_still_enforced (✅ FAILED as designed)
Flow YAML
id: qa_maxdepth_still_enforced
namespace: qa.transform
tasks:
- id: deep_non_tail_recursion
type: io.kestra.plugin.transform.jsonata.TransformValue
from: "{}"
maxDepth: 10
expression: |
(
$f := function($n) { $n <= 0 ? 0 : 1 + $f($n - 1) };
$f(500)
)Gantt
| Task | Status | Duration |
|---|---|---|
deep_non_tail_recursion |
FAILED | 0.08s |
| Total | FAILED | 0.20s |
Logs synthesis — the bound still fires on genuinely deep recursion, and the new message is the actionable one (confirmed rendered in the UI Logs tab, not just the API):
ERROR JSonataException JSONata expression exceeded maxDepth=10 nested evaluation levels.
Raise maxDepth if the expression legitimately recurses that deep, otherwise check for a
recursive function that never reaches its terminating case.
Stack trace originates at EvaluationBounds.entry → Transform.evaluateExpression(Transform.java:143). This replaces the old message that blamed recursion the user never wrote — the specific complaint in #102.
Flow 7: qa_timeout_still_enforced (✅ FAILED as designed — but the plugin's own message never surfaces)
Flow YAML
id: qa_timeout_still_enforced
namespace: qa.transform
tasks:
- id: long_running_tail_recursion
type: io.kestra.plugin.transform.jsonata.TransformValue
timeout: PT5S
maxDepth: 1000
from: "{}"
expression: |
(
$loop := function($n, $acc) { $n <= 0 ? $acc : $loop($n - 1, $acc + 1) };
$loop(500000000, 0)
)Gantt
| Task | Status | Duration |
|---|---|---|
long_running_tail_recursion |
FAILED | 5.04s |
| Total | FAILED | 5.13s |
Logs synthesis: fails at the 5s mark as intended — but with Kestra's worker timeout, not the plugin's:
ERROR Timeout after 00:00:05.000
io.kestra.core.exceptions.TimeoutExceededException at WorkerTaskCallable.doCall(WorkerTaskCallable.java:97)
EvaluationBounds' own message ("JSONata evaluation exceeded timeout=…ms") never appears. That follows from Transform.evaluateExpression deriving timeoutMillis from getTimeout() — the same task-level property the worker enforces — and the worker's clock starting earlier (task start) than the evaluation's (frame registration). The worker therefore always wins the race, so that message looks unreachable via the task property.
This is not a defect, and the timeout code is not dead. I verified it does real work: evaluation runs on a detached 4 MB-stack executor thread that the worker cannot interrupt, so without an internal bound the runaway loop would keep spinning after the task is marked failed. Container CPU across a fresh run:
| Moment | Container CPU |
|---|---|
| Baseline (idle) | 3.2% |
| t+3s, mid-evaluation | 143.0% |
| t+8s, just after the 5s timeout | 19.6% |
| t+23s | 6.1% |
The evaluation genuinely stops. Worth a one-line comment in EvaluationBounds that its timeout exists to stop the detached eval thread rather than to produce a user-facing error — otherwise the next reader will reasonably try to make that message appear and conclude it is broken.
Flow 8: qa_doc_guidance_rewrite (✅ SUCCESS)
Validates the guidance this PR adds to io.kestra.plugin.transform.jsonata.md.
Flow YAML
id: qa_doc_guidance_rewrite
namespace: qa.transform
tasks:
- id: gen_500
type: io.kestra.plugin.core.storage.Write
extension: .json
content: |
[{% for i in range(1, 500) %}{"id": {{ i }}, "sku": "SKU-{{ i }}", "qty": {{ i % 7 + 1 }}, "price": {{ i }}}{% if not loop.last %},{% endif %}{% endfor %}]
- id: recommended_path_projection
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.gen_500.uri }}"
expression: |
{
"items": [$.{
"order_id": sku,
"total": qty * price
}]
}
- id: reported_slow_shape_lambda_in_object
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.gen_500.uri }}"
expression: |
{
"items": $map($, function($r) {
{
"order_id": $r.sku,
"total": $r.qty * $r.price
}
})
}Gantt
| Task | Status | Duration |
|---|---|---|
gen_500 |
SUCCESS | 0.17s |
recommended_path_projection |
SUCCESS | 0.19s |
reported_slow_shape_lambda_in_object |
SUCCESS | 0.65s |
| Total | SUCCESS | 1.36s |
Outputs synthesis — the doc claims the rewrite is output-preserving. Verified directly: both output ION files are 15562 bytes and byte-for-byte identical (cmp clean). Sample:
{items:[{order_id:"SKU-1",total:2},{order_id:"SKU-2",total:6},{order_id:"SKU-3",total:12},…
At this width the slow shape is only ~3.4× slower (0.65s vs 0.19s), far from #102's 7976 ms at 500 items — again consistent with cost scaling in projection width, which my 2-key projection minimises. The doc caveat is directionally correct; this flow does not measure its worst case.
Flows 1–3: non-regression on the plugin's own @Example flows
All three run verbatim from the @Plugin(examples = …) blocks in TransformItems.java / TransformValue.java.
Flow YAML — all three
id: jsonata_example
namespace: company.team
tasks:
- id: http_download
type: io.kestra.plugin.core.http.Download
uri: https://dummyjson.com/products
- id: get_product_and_brand_name
description: "String Transformation"
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.http_download.uri }}"
expression: products.(title & ' by ' & brand)
- id: get_total_price
description: "Number Transformation"
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.http_download.uri }}"
expression: $sum(products.price)
- id: sum_up
description: "Writing out results in the form of JSON"
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.http_download.uri }}"
expression: |
{
"total_products": $count(products),
"total_price": $sum(products.price),
"total_discounted_price": $sum(products.(price-(price*discountPercentage/100)))
}
---
id: jsonata
namespace: company.team
tasks:
- id: extract
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/resolve/main/json/orders.json
- id: jsonata
type: io.kestra.plugin.transform.jsonata.TransformItems
from: "{{ outputs.extract.uri }}"
expression: |
Account.Order.{
"order_id": OrderID,
"total": $round($sum(Product.(Price * Quantity)), 2)
}
---
id: jsonata_transform_value
namespace: company.team
tasks:
- id: transform_json
type: io.kestra.plugin.transform.jsonata.TransformValue
from: |
{
"order_id": "ABC123",
"first_name": "John",
"last_name": "Doe",
"address": { "city": "Paris", "country": "France" },
"items": [
{ "product_id": "001", "name": "Apple", "quantity": 5, "price_per_unit": 0.5 },
{ "product_id": "002", "name": "Banana", "quantity": 3, "price_per_unit": 0.3 },
{ "product_id": "003", "name": "Orange", "quantity": 2, "price_per_unit": 0.4 }
]
}
expression: |
{
"order_id": order_id,
"customer_name": first_name & ' ' & last_name,
"address": address.city & ', ' & address.country,
"total_price": $sum(items.(quantity * price_per_unit))
}Gantt
| Flow | Task | Status | Duration |
|---|---|---|---|
jsonata_example |
http_download |
SUCCESS | 6.00s |
get_product_and_brand_name |
SUCCESS | 0.21s | |
get_total_price |
SUCCESS | 0.08s | |
sum_up |
SUCCESS | 0.05s | |
| Total | SUCCESS | 6.89s | |
jsonata |
extract |
SUCCESS | 0.59s |
jsonata |
SUCCESS | 0.09s | |
| Total | SUCCESS | 0.82s | |
jsonata_transform_value |
transform_json |
SUCCESS | 0.03s |
| Total | SUCCESS | 0.08s |
The 6.00s in http_download is dummyjson.com latency, not the plugin.
Outputs synthesis
jsonata_example:get_product_and_brand_name→processedItemsTotal: 30;get_total_priceandsum_up→processedItemsTotal: 1each, all with.ionURIs. Correct for these expressions.jsonata:processedItemsTotal: 5(5 orders inorders.json).jsonata_transform_value:value: {order_id: "ABC123", customer_name: "John Doe", address: "Paris, France", total_price: 4.2}— matches the example's intended result exactly (5×0.5 + 3×0.3 + 2×0.4 = 4.2).
Logs synthesis: no warnings or errors in any of the three beyond the harmless Input wait was provided… but isn't declared notice from how QA triggered the executions.
Notes
- Topology / plugin-artifact section skipped: this PR does not touch artifacts, there is no
ui/folder in the repo, andv1.3.30is below the 2.0 floor for plugin artifacts. - Screenshots not attached:
0x0.sthas uploads disabled ("almost nothing but AI botnet spam"). No substitute host was used. The instance is left running instead, so the Gantt / Logs / Outputs / Topology tabs can be inspected directly at the URL in the header. - Method: login and per-flow execute were driven through the UI with
agent-browser(flow 1 end-to-end, plus the Gantt/Logs/Outputs tabs for flows 4, 5, 6); the remaining executions were triggered over the API against the same instance and their results read back from the UI and the execution/log endpoints. Flow definitions were created via the flows API rather than typed into the Monaco editor. - No unit or integration tests were run in the plugin repository.
Follow-up review (correctness + quality)Agrees with @jymaire's APPROVE. Independently re-verified the load-bearing claims against the decompiled 1. 🟡
|
Review notes on #103, all non-blocking: - Disambiguate the maxDepth schema text. "a batch of 1,000,000 records needs no more than a batch of 10" elided its subject and first-read as a comparison of batch sizes rather than of the maxDepth each needs. - Split enter() into checkDepth()/checkTimeout(), and record why the timeout is checked on entry only where Timebox checked both sides. - Use {@code} rather than {@link} for setRuntimeBounds: the link resolves against 0.9.10 but would fail -Xdoclint on a dependency bump that renames it. - Note in shouldStillEnforceTimeout that its margin is deliberate and should be widened rather than the test deleted if CI ever flakes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks @Malaydewangan09 — addressed in e16d902. Four of the five applied, one declined with reasoning. 1. 2. 3. Entry-only timeout check — fixed; you are right that this deserved a note rather than diff-archaeology. Now stated on 4. 5. Nits — both taken.
|
Closes the first half of #102.
Frame.setRuntimeBoundsinstalls aTimeboxwhose entry and exit callbacks both skip accounting when the frame carriesisParallelCall. The two sides don't observe the same flag for a givenevaluate()call, so the counter gains one level per input item and never unwinds —maxDepthwas capping how many records a flow could process rather than how deeply its expression nests. A 1000-item batch failed on the defaultmaxDepth: 1000while reporting non-terminating recursion for expressions containing none, and the value a flow needed changed with the size of that run's data.The callback setters are public, so the accounting can be done correctly from here without waiting on upstream.
What changed
EvaluationBounds(new) — registers entry/exit callbacks that count unconditionally, which is safe because the Java port evaluates path steps sequentially. Also carries the timeout check thatsetRuntimeBoundsused to provide.Transform.evaluateExpression— one line:frame.setRuntimeBounds(...)→EvaluationBounds.register(...).JSONataInterface—maxDepthschema rewritten. It described a limit on "recursive calls", which is not what the old counter measured.io.kestra.plugin.transform.jsonata.md— documents thatmaxDepthis independent of input size, and steers users off the slow shape described below.Measured with the same expression at different input sizes:
Not fixed here
The second defect in #102 is in
dashjoin/jsonata-javaand cannot be fixed from the plugin. It needs both a user-defined lambda in$mapand an object constructor consuming its result — at 1000 items:{"items": [$.{...}]}— path projection{"items": $map($, $string)}— builtin fn, same wrapper$map($, function($r){...})— lambda, no wrapper{"items": $map($, function($r){...})}— the reported shapeIt is unrelated to the bounds machinery: with runtime bounds disabled entirely the slow shape still takes 37.6 s.
Users are not blocked while it is outstanding. On the reported expression at its 1000-item batch size, with this PR applied: 18 s as written, 18 ms rewritten as a path projection, byte-identical output. The plugin doc now carries that guidance. Only expressions that genuinely need a function value — a lambda passed to
$reduceor$sort, or keys built conditionally per item — have no rewrite available.Worth noting from the same measurement: the reported expression peaks at 11 nested levels. The
maxDepth: 20000the reporter had to set was purely an artefact of the leaking counter.Test plan
./gradlew build— 150 tests green across all three modules.Four new tests, each verified to fail with
setRuntimeBoundsrestored rather than passing vacuously:setRuntimeBoundsTransformItemsTest#shouldCollapseBatchIntoOneRecordOnDefaultMaxDepth[100]TransformItemsTest#shouldCollapseBatchIntoOneRecordOnDefaultMaxDepth[5000]Depth=1001 max=1000, the reported errorTransformValueTest#shouldRequireTheSameMaxDepthRegardlessOfInputSize[10, 400]maxDepth=12TransformValueTest#shouldReportMaxDepthBreachWithActionableMessageTransformValueTest#shouldStillEnforceTimeoutThe timeout test uses a tail-recursive expression so TCO holds it at ~5 nested levels: it runs long purely through iteration count and so cannot pass by hitting the depth bound instead.
All six existing
StackOverflowError/ eval-thread regression tests from #79 and #80 pass unchanged — the 4 MB eval thread is untouched.🤖 Generated with Claude Code