Skip to content

Commit de5c162

Browse files
committed
fix(python): remove hardcoded OTEL_LOG_LEVEL=DEBUG that caused stderr deadlocks
Per #3357, `python.runScript()` deadlocks any Python subprocess that writes more than ~64KB to stderr. The previous implementation hardcoded `OTEL_LOG_LEVEL: 'DEBUG'` in the subprocess environment, which caused OTEL-aware Python libraries (mlflow, opentelemetry-sdk, etc.) to emit verbose debug-level logs to stderr during import. Once stderr exceeded the OS pipe buffer, the Python process would block on its next `write()` syscall indefinitely — a silent, permanent hang with no error or timeout (unless `maxDuration` was set). Remove the hardcoded `OTEL_LOG_LEVEL: 'DEBUG'`. This brings `runScript()` in line with `run()`, `runInline()`, and the streaming variants — none of which set `OTEL_LOG_LEVEL`. The original line was almost certainly leftover dev instrumentation that shipped accidentally. Closes #3357
1 parent 097fab0 commit de5c162

File tree

2 files changed

+5
-1
lines changed

2 files changed

+5
-1
lines changed
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@trigger.dev/python": patch
3+
---
4+
5+
Fix `python.runScript()` deadlock when the Python subprocess produces more than ~64KB of stderr output. The previous implementation hardcoded `OTEL_LOG_LEVEL: "DEBUG"` in the subprocess environment, which caused OTEL-aware Python libraries (mlflow, opentelemetry-sdk, etc.) to emit verbose debug-level logging to stderr during import. Once stderr exceeded the OS pipe buffer, the Python process would block on `write()` indefinitely. Removing the hardcoded debug log level brings `runScript()` in line with `run()`, `runInline()`, and the streaming variants — none of which set `OTEL_LOG_LEVEL`.

packages/python/src/index.ts

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,6 @@ export const python = {
9797
}=trigger,${Object.entries(taskContext.attributes)
9898
.map(([key, value]) => `${key}=${value}`)
9999
.join(",")}`,
100-
OTEL_LOG_LEVEL: "DEBUG",
101100
},
102101
},
103102
throwOnError: false,

0 commit comments

Comments
 (0)