Skip to content

Conversation

@MySweetEden
Copy link
Contributor

Motivation

PySCIPOpt already has recipe(s) that store optimization progress in memory. However, in-memory traces are not suitable for real-time, external observation (e.g., another process tailing progress, dashboards, log collectors).

This recipe focuses on the missing piece: a stream-friendly, structured output that can be consumed outside the running Python process during solve.

Design Decisions

  • JSONL format: Designed for streaming writes and partial reads; remains readable even if the run is interrupted or crashes
  • Real-time external output is the primary value:
    • Records progress updates as one JSON object per line
    • Flushes on key events so downstream consumers can react immediately
  • Schema compatibility with setTracefile() (PR Add setTracefile() method for structured optimization progress loggingAdd settracefile api #1158):
    • Uses the same JSONL field names (type, time, primalbound, dualbound, gap, nodes, nsol) for consistency across tracing approaches.
  • In-memory + file output: Keeps model.data["trace"] for convenience/testing, but the recipe is centered on file streaming via path=...
  • Robust termination signaling for external monitoring:
    • Always emits a final run_end record on normal termination, interruption, or exception
    • On exceptions, run_end includes structured error metadata (status, exception type, message)
    • Flushes run_end to make completion detection reliable

Events Recorded

  • bestsol_found: when a new best solution is found
  • dualbound_improved: when the dual bound improves
  • run_end: when optimization terminates (also emitted on interrupt/exception)

Fields

type, time, primalbound, dualbound, gap, nodes, nsol (aligned with the JSONL trace schema introduced in PR #1158)
(run_end may additionally include: status, exception, message on failure)

MySweetEden and others added 13 commits January 20, 2026 03:57
… handling; rename optimize_with_trace to optimizeTrace for clarity
…nified event writing method, improving clarity and consistency in event handling.
…ction, enhancing test coverage for both optimizeTrace and optimizeNogilTrace. Update assertions for trace data consistency.
…tracking

This update introduces a comprehensive docstring for the _TraceRun class, detailing its purpose, arguments, return values, and usage examples. This enhancement improves code documentation and usability for future developers.
…racking with JSONL output

This commit introduces the realtime_trace_jsonl recipe, which allows for real-time tracking of optimization progress and outputs the data in JSONL format. Additionally, the CHANGELOG has been updated to reflect this new feature.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant