Add realtime_trace_jsonl recipe for structured real-time optimization progress streaming #1177
Conversation
… handling; rename optimize_with_trace to optimizeTrace for clarity
…nified event writing method, improving clarity and consistency in event handling.
…ction, enhancing test coverage for both optimizeTrace and optimizeNogilTrace. Update assertions for trace data consistency.
…tracking This update introduces a comprehensive docstring for the _TraceRun class, detailing its purpose, arguments, return values, and usage examples. This enhancement improves code documentation and usability for future developers.
…racking with JSONL output This commit introduces the realtime_trace_jsonl recipe, which allows for real-time tracking of optimization progress and outputs the data in JSONL format. Additionally, the CHANGELOG has been updated to reflect this new feature.
…uments for clarity
Joao-Dionisio
requested review from
Joao-Dionisio and
Copilot
and removed request for
Copilot
There was a problem hiding this comment.
Pull request overview
Adds a new PySCIPOpt recipe to stream structured optimization progress in real time using JSONL, enabling external tailing/monitoring while the solver runs.
Changes:
- Introduces
realtime_trace_jsonlrecipe withoptimizeTrace()/optimizeNogilTrace()to record selected SCIP events intomodel.data["trace"]and optionally a JSONL file. - Records
bestsol_found,dualbound_improved, and a finalrun_endevent with flushing intended for real-time consumption. - Adds tests covering in-memory tracing, file output, and interrupt handling; updates changelog.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| src/pyscipopt/recipes/realtime_trace_jsonl.py | Implements the real-time JSONL tracing recipe and event handling. |
| tests/test_recipe_realtime_trace_jsonl.py | Adds tests for in-memory traces, JSONL file output, and interruption behavior. |
| CHANGELOG.md | Documents the addition of the new recipe. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| self._handler = _TraceEventhdlr() | ||
| self.model.includeEventhdlr( | ||
| self._handler, "realtime_trace_jsonl", "Realtime trace jsonl handler" | ||
| ) |
There was a problem hiding this comment.
includeEventhdlr() registers an event handler plugin permanently (there is no corresponding remove/uninclude API). Calling optimizeTrace()/optimizeNogilTrace() multiple times on the same model will attempt to include another handler with the same name (realtime_trace_jsonl), which can raise a SCIP error and/or leave multiple live handlers capturing closed file handles and old _TraceRun instances. Refactor to include the handler at most once per model (e.g., stash/reuse it in model.data), and make the handler read its current sink (trace list / file handle) from mutable attributes rather than a closure over a per-run object.
| if self._handler is not None: | ||
| for et in ( | ||
| SCIP_EVENTTYPE.BESTSOLFOUND, | ||
| SCIP_EVENTTYPE.DUALBOUNDIMPROVED, | ||
| ): | ||
| try: | ||
| self.model.dropEvent(et, self._handler) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
dropEvent() decrements the model refcount unconditionally (see Model.dropEvent in scip.pxi), so calling it from __exit__ is unsafe if eventinit() never executed or if the events were not actually caught (it can underflow the INCREF/DECREF pairing). A safer pattern is to implement eventexit() on the event handler to drop exactly the events it caught (and only after eventinit() ran), and remove the unconditional dropEvent() calls from this context manager.
| self._write_event( | ||
| "dualbound_improved", fields=snapshot, flush=False | ||
| ) |
There was a problem hiding this comment.
For a recipe marketed as “real-time JSONL streaming”, not flushing dualbound_improved events can delay visibility for external consumers tailing the file. Consider flushing here as well (or making flushing policy configurable), especially since dualbound_improved is one of the primary progress signals you record.
| def __enter__(self): | ||
| if not hasattr(self.model, "data") or self.model.data is None: | ||
| self.model.data = {} | ||
| self.model.data.setdefault("trace", []) |
There was a problem hiding this comment.
model.data.setdefault("trace", []) will append to an existing trace from previous runs on the same model, while the file output is truncated (open(..., "w")). To keep in-memory and file behavior consistent (and avoid mixing multiple runs), reset model.data["trace"] to a new empty list at the start of each traced run.
| self.model.data.setdefault("trace", []) | |
| # Start a fresh trace for each run to keep in-memory and file behavior consistent | |
| self.model.data["trace"] = [] |
| def eventinit(s): | ||
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, s) | ||
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, s) | ||
|
|
||
| def eventexec(s, event): |
There was a problem hiding this comment.
Normal methods should have 'self', rather than 's', as their first parameter.
| def eventinit(s): | |
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, s) | |
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, s) | |
| def eventexec(s, event): | |
| def eventinit(self): | |
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, self) | |
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, self) | |
| def eventexec(self, event): |
| def eventinit(s): | ||
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, s) | ||
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, s) | ||
|
|
||
| def eventexec(s, event): |
There was a problem hiding this comment.
Normal methods should have 'self', rather than 's', as their first parameter.
| def eventinit(s): | |
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, s) | |
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, s) | |
| def eventexec(s, event): | |
| def eventinit(handler): | |
| self.model.catchEvent(SCIP_EVENTTYPE.BESTSOLFOUND, handler) | |
| self.model.catchEvent(SCIP_EVENTTYPE.DUALBOUNDIMPROVED, handler) | |
| def eventexec(handler, event): |
| ): | ||
| try: | ||
| self.model.dropEvent(et, self._handler) | ||
| except Exception: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
|
I’ll address the comments over the weekend and push updates soon. |
1 participant
Motivation
PySCIPOpt already has recipe(s) that store optimization progress in memory. However, in-memory traces are not suitable for real-time, external observation (e.g., another process tailing progress, dashboards, log collectors).
This recipe focuses on the missing piece: a stream-friendly, structured output that can be consumed outside the running Python process during solve.
Design Decisions
setTracefile()(PR Add setTracefile() method for structured optimization progress loggingAdd settracefile api #1158):type,time,primalbound,dualbound,gap,nodes,nsol) for consistency across tracing approaches.model.data["trace"]for convenience/testing, but the recipe is centered on file streaming viapath=...run_endrecord on normal termination, interruption, or exceptionrun_endincludes structured error metadata (status, exception type, message)run_endto make completion detection reliableEvents Recorded
bestsol_found: when a new best solution is founddualbound_improved: when the dual bound improvesrun_end: when optimization terminates (also emitted on interrupt/exception)Fields
type, time, primalbound, dualbound, gap, nodes, nsol(aligned with the JSONL trace schema introduced in PR #1158)(
run_endmay additionally include:status, exception, messageon failure)