[core] fix: wrap TQ write in try-except to prevent training hang on write failure

[Erpim]

What does this PR do?

Description

Problem

When _write_session_trajectories_to_tq raises an exception during TransferQueue write (e.g., KeyError if required fields like finish_reason are missing from the trajectory), the exception propagates out of _run_prompt_sessions_to_tq before it can update the prompt's status in TQ ("finished" / "failure").

This leaves the prompt permanently in "pending" state, causing _has_enough_samples in the replay buffer to never reach batch_size. The training loop hangs indefinitely, with no visible error other than a cryptic failure_reason: "'finish_reason'" in the logs (which loses the exception class name).

Root cause flow

1. _run_session returns trajectories successfully
2. _write_session_trajectories_to_tq raises (e.g. KeyError)
3. Exception propagates out of _run_prompt_sessions_to_tq → captured by asyncio.gather(return_exceptions=True) in _run_batch_to_tq
4. _short_failure_reason calls str(KeyError('finish_reason')) which produces "'finish_reason'" — the exception type is lost
5. Prompt remains "pending" in TQ → replay_buffer.sample() blocks forever

Checklist Before Starting

• Search for similar PRs or issues and paste at least one relevant link here: ...
• Format the PR title as [{modules}] {type}: {description} (checked by CI)
  • {modules} may include core, interaction, model, env, tools, deployment, reward, dashboard, docs, examples, data, train, ci, build, deps, misc
  • If this PR involves multiple modules, separate them with , like [interaction, tools, docs]
  • {type} must be one of feat, fix, refactor, chore, test
  • If this PR breaks an API, config contract, workflow, or other compatibility boundary, add [BREAKING] to the beginning of the title
  • For a stacked PR series, you may prepend a progress marker such as [1/N]
  • Example: [BREAKING][deployment, docs] feat: simplify runtime env configuration

Test

Defense against abnormal scenarios.

API and Usage Example

N/A

Design & Code Changes

Fix

1. _write_session_trajectories_to_tq: Wrap the call in try-except. On failure, log the full traceback via logger.exception, count the session as failed, and append the real error to
   failure_reasons. Move success_sessions and success_outputs to the else branch so they only increment on successful TQ write.
2. _short_failure_reason: Include the exception class name in the output, so "'finish_reason'" becomes "KeyError: 'finish_reason'" for easier debugging.

This ensures:

• Full traceback is logged for TQ write failures
• Prompt status is properly updated — no more stuck "pending" prompts
• Real error message appears in generate_sequences summary's failure_reasons
• Training continues instead of hanging silently

Checklist Before Submitting

• Read the Contribute Guide
• Run pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add or update docs/examples for user-facing changes
• Add tests or explain why tests are not practical
• Confirm the PR title matches the required format
• Confirm the placeholder text in this template has been replaced with real content

[gemini-code-assist]

Code Review

This pull request updates _short_failure_reason to include the exception class name in the returned string and wraps the _write_session_trajectories_to_tq call in a try-except block to handle and log write failures. The review comments suggest avoiding redundant class names when the exception message is empty and using the _short_failure_reason helper when appending TQ write errors to failure_reasons.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If the exception message is empty (e.g., for a generic exception without a message), message is set to error.__class__.__name__. This results in a redundant return value of ClassName:ClassName (e.g., ValueError:ValueError). We can simplify this by returning just the class name when the message is empty.

Suggested change

	message = str(error)
	if not message:
	message = error.__class__.__name__
	return message[:512]
	return f"{error.__class__.__name__}:{message}"[:512]
	message = str(error)
	if not message:
	return error.__class__.__name__[:512]
	return f"{error.__class__.__name__}:{message}"[:512]

[gemini-code-assist]

Since the goal of this PR is to include the exception class name in the failure reasons for easier debugging, we should use the _short_failure_reason helper here as well. Otherwise, f"TQ write error: {e}" will still lose the exception class name (e.g., producing TQ write error: 'finish_reason' instead of TQ write error: KeyError:'finish_reason').

Suggested change

	except Exception as e:
	logger.exception(f"TQ write failed for uid={uid} session={session_index}: {e}")
	failed_sessions += 1
	failure_reasons.append(f"TQ write error: {e}")
	except Exception as e:
	logger.exception(f"TQ write failed for uid={uid} session={session_index}: {e}")
	failed_sessions += 1
	failure_reasons.append(f"TQ write error: {_short_failure_reason(e)}")