Mondaic
This API reference is not for the latest stable Salvus version.

salvus.flow.sites.task_chain.task_chain

Classes

TaskChain

class TaskChain(builtins.object):
    def __init__(
        self,
        site: salvus.flow.sites.base_site.BaseSite,
        tasks: List,
        ranks: int,
        wall_time_in_seconds: Optional[
            int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64
        ] = None,
        data_files_to_upload: Optional[List[Union[pathlib.Path, str]]] = None,
        job_name: Optional[str] = None,
        initialize_on_site: bool = True,
        python_call_prefix: Optional[str] = None,
        verbosity: int = 1,
    ):
        ...

A chain of tasks to be run on a remote site.

Each task can either be a Salvus simulation or a Python function.

Parameters
  • site salvus.flow.sites.base_site.BaseSite — The site this job is to be run at.
  • tasks List — A list of tasks.
  • ranks int — Number of ranks for the Salvus simulations.
  • wall_time_in_seconds Optional[int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64] — Wall time in seconds for the job. Not required for jobs on all sites.
  • data_files_to_upload Optional[List[Union[pathlib.Path, str]]] — List of optional data files to upload to the task chain directory.
  • job_name Optional[str] — Name of the job.
  • initialize_on_site bool — Create the remote folder structure and upload all input files. Set to False to recreate a TaskChain object that has already been initialized once.
  • python_call_prefix Optional[str] — Optional call prefix for the Python calls - useful if they for example have to be spawned with special MPI calls.
  • verbosity int — The verbosity level during the initialization.
Attributes
full_name str

The full name of the task chain including the site name.

job_name str

The task chain’s name.

path Union[pathlib.PurePosixPath, pathlib.PureWindowsPath]

Path of the task chain.

stderr Dict[Union[str, int], str]

Stderr by task.

stdout Dict[Union[str, int], str]

Stdout by task.

task_chain_status_file Optional[Dict]

Get the contents of the task chain status file.

task_status Dict[int, salvus.flow.sites.types.JobStatus]

Get the status of the individual tasks.

timings Dict

Timings for the individual tasks.

tmp_path Union[pathlib.PurePosixPath, pathlib.PureWindowsPath]

Temporary folder.

working_dir Union[pathlib.PurePosixPath, pathlib.PureWindowsPath]

Working directory of the task chain.

Methods
cancel()
def cancel(self, verbosity: int = 1) -> None:
    ...

Cancel the task chain.

Parameters
  • verbosity int — Verbosity level.
Returns None
delete()
def delete(
    self, verbosity: int = 1, allow_directory_deletion_failure: bool = False
) -> None:
    ...

Deletes the data of a task chain if it is not running or pending.

Will delete all remote files and the internal database entry.

Parameters
  • verbosity int — Verbosity level.
  • allow_directory_deletion_failure bool — If True this function will still return successfully (but raise a warning) if the deletion of the remote Salvus managed files failed for some reason. This can easily happen on for example networked file systems.
Returns None
get_diagnosis_string()
def get_diagnosis_string(self) -> str:
    ...

Get a hopefully useful diagnostic string from the job array.

Useful for trying to figure out why a task chain fails.

Returns str
launch()
def launch(self, verbosity: int = 1, interactive_tty: bool = False) -> None:
    ...

Launch the job.

Parameters
  • verbosity int — Verbosity level.
  • interactive_tty bool — Connect an interactive terminal. Useful for debugging the Python scripts. Only works for local sites.
Returns None
show_pretty_status_widget()
def show_pretty_status_widget(self) -> None:
    ...

Show a pretty status widget to quickly inspect what goes happened in a task chain.

Returns None
tail_stdout()
def tail_stdout(self) -> None:
    ...

tail -f the stdout of the task chain.

This will launch a remote tail command and it only suitable for interactive use.

Returns None
update_status()
def update_status(
    self,
    progress_callback: Optional[Callable[[Dict], NoneType]] = None,
    force_update: bool = False,
) -> salvus.flow.sites.types.JobStatus:
    ...

Update the status of the task chain by interacting with the remote site.

Parameters
  • progress_callback Optional[Callable[[Dict], NoneType]] — Callback function which will receive some information about the update.
  • force_update bool — Jobs that are not running and not pending will by default only query the local job database and not check the remote site. In most cases this is correct as the jobs are done with and will not change again. Setting this to True will force an update nonetheless. Uses for this are for example if a job update fails for example due to a temporary network problem.
Returns salvus.flow.sites.types.JobStatus
wait()
def wait(
    self,
    poll_interval_in_seconds: Optional[
        int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64
    ] = None,
    timeout_in_seconds: Optional[float] = None,
    verbosity: int = 1,
    progress_callback: Optional[Callable[[Union[Dict, str]], NoneType]] = None,
) -> salvus.flow.sites.types.JobStatus:
    ...

Wait for the job to finish.

Parameters
  • poll_interval_in_seconds Optional[int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64] — If given, the site will be queried with this interval. Defaults to the site’s default.
  • timeout_in_seconds Optional[float] — Time in seconds after which this function will return, independent of the status.
  • verbosity int — The verbosity level.
  • progress_callback Optional[Callable[[Union[Dict, str]], NoneType]] — Optional progress callback function. Will be called if there are any status updates. Will currently only be called with the Salvus status file contents.
Returns salvus.flow.sites.types.JobStatus