Mondaic

salvus.flow.ng.in_process_job

salvus.flow.ng.in_process_job salvus flow ng in_process_job

Job-like wrapper around in-process runs.

Functions

run_in_process()

def run_in_process(
    solver: Solver,
    input_file: typing.Any,
    output_folder: str | pathlib.Path | None = None,
    overwrite: bool = False,
    job_name: str | None = None,
    ranks: int | None = None,
    verbosity: int = 1,
    telemetry_interval_seconds: float = 2.0,
    wait: bool = True,
    progress: FrontendOverride = "auto",
) -> NGInProcessJob: ...

Run a simulation in-process with the solver.

The backend of salvus.flow.functions.run and run_async for Solver site names. run() calls this with wait=True (the default): synchronous, as before T2 — the returned job has already finished. run_async() calls this with wait=False: the solve runs on a daemon worker thread and this function returns the NGInProcessJob immediately, before the run finishes — poll job.update_status()/job.telemetry(), iterate job. telemetry_stream(), or call job.wait() to block until it does. Either way, a live progress display (see salvus.flow.ng.progress_display) starts immediately and runs on its own daemon thread — wait=True still shows it, since this function itself always runs the solve on a worker thread internally and only blocks (job.wait()) once the display has started; the native session bindings release the GIL during advance()/run_segment() so that display thread keeps rendering while a solve chunk runs.

A TelemetryCollector and, whenever output_folder is given, a telemetry.FileSink AND a telemetry.ZmqSink are always built (cheap: an in-memory dataclass swap plus, at most, one small JSON write every telemetry_interval_seconds, plus one daemon thread idling on a socket poll) — see notes/ng-run-telemetry-design.md (control-project repo) for the design this implements (T1+T2). progress.json is written into output_folder continuously while the run is in progress, not only once at the end, so a process watching the folder from outside sees live progress; the ZmqSink’s own telemetry_endpoint.json lets a separate Python process on the same machine query (snapshot/history/ping) and cancel this run directly, without polling files. Without an output_folder telemetry still runs in-memory only (no sinks — no folder to write progress.json/telemetry_endpoint.json into), so job.telemetry()/job.cancel() still work for a same-process caller, just not for a separate process.

Parameters
  • solver Solver — The runtime solver configuration.
  • input_file typing.Any — The simulation object. Currently this must be a simple_config.simulation.Waveform instance with sources, receivers, and a mesh attached.
  • output_folder str | pathlib.Path | None — If given, receivers.h5, meta.json, progress.json, and telemetry_endpoint.json are written there.
  • overwrite bool — Allow writing into an already existing output folder. Existing files are not removed first, matching the command-line solver run behavior.
  • job_name str | None — An optional custom job name. Also becomes progress.json’s own telemetry.run.job_id.
  • ranks int | None — Accepted for compatibility; the solver runs single-process, so anything but 1 warns.
  • verbosity int — The verbosity level.
  • telemetry_interval_seconds float — Target spacing, in seconds, between progress.json rewrites and between adaptive-chunking progress_callback firings (solve_forward_through_sessions’s own progress_interval_seconds) — see telemetry.FileSink.
  • wait bool — True (the default) blocks until the solve finishes and returns the finished job, as before T2 (internally, the solve still runs on a worker thread — this is what lets a live display run at the same time; see above). False returns the NGInProcessJob immediately; job.wait() blocks until it finishes.
  • progress FrontendOverride — "auto" (the default) resolves a live display via progress_display.detect_frontend() (an ipywidgets panel in Jupyter, a rich TUI panel on an interactive terminal, a tqdm bar otherwise); "widget"/"panel"/"tqdm" force that frontend; "plain" shows no live display but leaves the old, verbosity-gated print lines (P6) alone; "none" shows nothing at all, including those print lines, regardless of verbosity.
Returns NGInProcessJob — The job — already finished if wait=True, still running if wait=False.

Classes

NGInProcessJob

class NGInProcessJob(builtins.object):
    def __init__(
        self,
        event_data: EventData | None = None,
        recorded_data: typing.Any = None,
        event_block: EventBlock | None = None,
        time_axis: typing.Any = None,
        runtime_in_seconds: float | None = None,
        job_name: str | None = None,
        boundaries: typing.Sequence[dict] | None = None,
        timing_information: dict | None = None,
        telemetry_snapshot: TelemetrySnapshot | None = None,
        collector: TelemetryCollector | None = None,
    ) -> None: ...

The result of an in-process run, mimicking a SalvusJob.

Returned by salvus.flow.functions.run (and run_async) when a Solver object is passed instead of a site name. run_async (run_in_process(..., wait=False)) runs the solve on a worker thread and returns this object immediately, before the run has finished — wait()/update_status() reflect the run’s real, live status via the same TelemetryCollector the solve loop updates; get_as_event()/copy_output() raise until the run actually finishes. run() (the default, wait=True) keeps the pre-T2 behavior: the run has already finished by the time this object exists. Either way this is not backed by the job database.

Wrap an in-process run, finished or still running.

Parameters
  • event_data EventData | None — The recorded seismograms. None until the run finishes (always the case at construction for an async, wait=False run).
  • recorded_data typing.Any — The raw recorded receiver values, shape [n_time_steps + 1, n_receivers, n_components]. None until the run finishes.
  • event_block EventBlock | None — The event block defining sources and receivers. None until the run finishes.
  • time_axis typing.Any — The ng_salvus time axis of the run. None until the run finishes.
  • runtime_in_seconds float | None — Wall time of the solve. None until the run finishes.
  • job_name str | None — An optional custom job name.
  • boundaries typing.Sequence[dict] | None — The boundary conditions the solver applied (NGSetup.boundaries), recorded in meta.json.
  • timing_information dict | None — A serialized Timer tree, as _serialize_timer produces.
  • telemetry_snapshot TelemetrySnapshot | None — The run’s final TelemetrySnapshot, for a job constructed without a live collector (e.g. directly, in a test) — update_status() falls back to mapping this when collector is None.
  • collector TelemetryCollector | None — The run’s live TelemetryCollector, when one exists (every run through run_in_process builds one). update_status(), telemetry(), telemetry_stream(), and cancel() all go through this directly rather than the telemetry_snapshot fallback above, since it is always at least as current.
Methods
cancel()
def cancel(self) -> None: ...

Request cancellation of a still-running async (wait=False) run — honored at the next progress-callback chunk boundary (see solve_forward_through_sessions’s own cancel_event parameter), not immediately. A no-op if the run has already finished, or has no live collector at all (e.g. a synchronous run, which has already finished by the time this object exists).

A separate process does the equivalent by sending {"op": "cancel"} to this run’s ZmqSink (see telemetry_endpoint.json) — both paths set the exact same threading.Event.

Returns None
copy_output()
def copy_output(
    self,
    destination: str | pathlib.Path,
    allow_existing_destination_folder: bool = False,
    status: str | None = None,
) -> tuple[dict[str, int], bool]: ...

Write the run’s output files to a folder.

Produces the same receivers.h5, meta.json, and job_info.json files a SalvusCompute run would leave behind; a project only treats a simulation as computed once job_info.json exists.

Parameters
  • destination str | pathlib.Path — The output folder.
  • allow_existing_destination_folder bool — Allow writing into an existing folder.
  • status str | None — Written under meta.json’s own status key. None (the default) uses the freshest known TelemetrySnapshot.run.status ("finished" if none is known at all) — run_in_process passes this explicitly instead, since it calls this method before the collector’s own final snapshot (which would otherwise still read "running") is built.
Returns tuple[dict[str, int], bool] — Tuple of (file name to file size map, completion flag).
delete()
def delete(self) -> None: ...

Do nothing; there are no remote files or database entries.

Returns None
display()
def display(self, kind: FrontendOverride = "auto") -> DisplayHandle | None: ...

Start a live, environment-adapted progress display for this job (an ipywidgets panel in Jupyter, a rich TUI panel on an interactive terminal, a tqdm bar otherwise — see salvus.flow.ng.progress_display), fed from telemetry_stream() on its own daemon thread.

Safe to call on a job at any point in its life: already finished (renders the final snapshot once, synchronously, and starts no thread), still running (streams live until a terminal snapshot arrives, then self-closes), or one built with no live collector at all (same one-shot behavior as the finished case, via telemetry_stream()’s own fallback).

Parameters
  • kind FrontendOverride — "auto" (the default) resolves via progress_ display.detect_frontend(); otherwise "widget", "panel", "tqdm", "plain", or "none" forces that frontend.
Returns DisplayHandle | None — The running DisplayHandle, or None for kind in ("plain", "none") — both mean “no live display.”
get_as_event()
def get_as_event(self) -> EventData: ...

Get the simulation results.

Returns EventData — The recorded seismograms as an EventData object.
get_diagnosis_string()
def get_diagnosis_string(self) -> str: ...

Get a diagnosis string for the job.

Returns str
get_status_from_db()
def get_status_from_db(self) -> JobStatus: ...

Return the current status — matches command-line solver SalvusJob. get_status_from_db() in shape (for job-agnostic callers that read it unconditionally, e.g. SimulationStore._to_info_dict), but there is no database behind an in-process run: this simply returns update_status()’s own always-live result.

Returns JobStatus — The mapped JobStatus.
telemetry()
def telemetry(self) -> TelemetrySnapshot | None: ...

The freshest known TelemetrySnapshot for this run, or None if none has been recorded yet (immediately after an async construction, before the worker thread’s first publish).

This same-process job object reads its own TelemetryCollector directly — the cheapest and always-current path. A separate process on the same machine reaches the identical live state via the ZmqSink/telemetry_endpoint.json handshake this run’s run_in_process call also sets up (see telemetry.ZmqSink), not through this method.

Returns TelemetrySnapshot | None
telemetry_stream()
def telemetry_stream(
    self, poll_interval_seconds: float = 0.1
) -> typing.Iterator[TelemetrySnapshot]: ...

Yield each new TelemetrySnapshot as it is recorded, until the run reaches a terminal status (finished/failed/ cancelled), then stop.

Polls the live collector (there is no push notification in-process) at poll_interval_seconds; for a job with no collector at all (e.g. constructed directly in a test), any telemetry_snapshot given at construction is yielded once, then the generator ends immediately.

Parameters
  • poll_interval_seconds float — How often to check for a new snapshot.
Returns typing.Iterator[TelemetrySnapshot]
update_status()
def update_status(self) -> JobStatus: ...

Return the job status.

Maps the freshest known TelemetrySnapshot.run.status (the live collector’s own latest, when one is attached — which every run through run_in_process does — otherwise the telemetry_snapshot a job was directly constructed with) onto a JobStatus; falls back to JobStatus.finished when neither exists (e.g. a job constructed with no telemetry at all, in a test), matching this method’s behavior before telemetry existed.

Returns JobStatus — The mapped JobStatus.
wait()
def wait(self, timeout: float | None = None) -> JobStatus: ...

Block until the run finishes (a no-op if it already has), then return the job status.

Parameters
  • timeout float | None — Maximum seconds to wait for the worker thread, as threading.Thread.join. None (the default) waits indefinitely. Has no effect on a synchronous (wait=True) run, which has already finished.
Returns JobStatus — The mapped JobStatus — see update_status().