Error chunks are currently hand built JSON strings with inconsistent schemas across IORails and LLMRails (different fields, different type/code values, LLMRails includes "param" while IORails doesn't). The API consumer layer has no contract to parse against.
Worse, because errors are plain JSON strings, they're indistinguishable from LLM output and flow into RollingBuffer and get checked by output rails as content.
Fix: add a StreamError datamodel to a shared module. Both IORails and LLMRails construct StreamError objects instead of inline json.dumps. StreamingHandler yields them as typed objects. Consumer uses isinstance instead of JSON parsing heuristics. RollingBuffer skips them.
class StreamErrorDetails(BaseModel):
message: str
type: str
code: str
param: str | None = None # flow_id, when applicable
class StreamError(BaseModel):
error: StreamErrorDetails
class StreamErrorType(str, Enum):
GENERATION_ERROR = "generation_error"
GUARDRAILS_VIOLATION = "guardrails_violation"
INTERNAL_ERROR = "internal_error"
class StreamErrorCode(str, Enum):
GENERATION_FAILED = "generation_failed"
CONTENT_BLOCKED = "content_blocked"
RAIL_EXECUTION_FAILURE = "rail_execution_failure"The enums enforce consistent vocabulary. All existing call sites in both IORails and LLMRails need to be migrated to use these instead of inline strings. LLMRails extract_error_json() should also be updated to produce a StreamError with valid enum values rather than passing through arbitrary upstream error shapes.
Should land after #1766.
Error chunks are currently hand built JSON strings with inconsistent schemas across IORails and LLMRails (different fields, different type/code values, LLMRails includes "param" while IORails doesn't). The API consumer layer has no contract to parse against.
Worse, because errors are plain JSON strings, they're indistinguishable from LLM output and flow into RollingBuffer and get checked by output rails as content.
Fix: add a StreamError datamodel to a shared module. Both IORails and LLMRails construct StreamError objects instead of inline json.dumps. StreamingHandler yields them as typed objects. Consumer uses isinstance instead of JSON parsing heuristics. RollingBuffer skips them.
The enums enforce consistent vocabulary. All existing call sites in both IORails and LLMRails need to be migrated to use these instead of inline strings. LLMRails
extract_error_json()should also be updated to produce a StreamError with valid enum values rather than passing through arbitrary upstream error shapes.Should land after #1766.