Mondaic
This API reference is not for the latest stable Salvus version.

salvus.flow.sites.job

Classes

Job

class Job(builtins.object):
    def __init__(
        self,
        site: salvus.flow.sites.base_site.BaseSite,
        commands: List[salvus.flow.sites.site_utils.RemoteCommand],
        job_type: str,
        job_description: str,
        job_info: Dict,
        jobname: Optional[str] = None,
        jobname_postfix: str = "",
        is_debug_job: bool = False,
        wall_time_in_seconds: Optional[
            int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64
        ] = None,
        job_groups: Optional[List[Union[str, salvus.flow.db.JobGroup]]] = None,
        rundir_root: Optional[
            str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath
        ] = None,
        tmpdir_root: Optional[
            str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath
        ] = None,
        working_dir: Optional[
            str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath
        ] = None,
        allow_uninitialized_site: bool = False,
        requires_tmp_dir: bool = False,
        initialize_on_site: bool = True,
        no_db: bool = False,
    ):
        ...

A class representing a concrete job associated with a certain site.

A job is defined as a series of commands that require interaction with the job scheduler.

All interactions like updating the job or persisting some things to the database should flow over this class.

Parameters
  • site salvus.flow.sites.base_site.BaseSite — The site this job is to be run at.
  • commands List[salvus.flow.sites.site_utils.RemoteCommand] — A list of remote command objects.
  • job_type str — The type of job.
  • job_description str — Short textual description about the text.
  • job_info Dict — Additional information about the job.
  • jobname Optional[str] — Name of the job. A random one will be created if none is given.
  • jobname_postfix str — Optional postfix string for the job name.
  • is_debug_job bool — True if this job should be executed in the fast/debug queue if the site has one.
  • wall_time_in_seconds Optional[int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64] — Wall time in seconds for the job. Not required for jobs on all sites.
  • job_groups Optional[List[Union[str, salvus.flow.db.JobGroup]]] — List of job groups names this job should be part of.
  • rundir_root Optional[str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath] — Specify what parent folder the run directory will be created in. Defaults to the directory in the site config.
  • tmpdir_root Optional[str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath] — Specify what parent folder the tmp directory will be created in. Defaults to the directory in the site config.
  • working_dir Optional[str, pathlib.Path, pathlib.PurePosixPath, pathlib.PureWindowsPath] — Working directory. If not given it will be set to the job directory. Should only be set if the job is part of a job array.
  • allow_uninitialized_site bool — Allows an uninitialized site - intended to only be used for the job that initializes the site.
  • requires_tmp_dir bool — The job does require a temporary directory.
  • initialize_on_site bool — Initialize on site. Should be true in most cases. False is useful for creating a Job object from the database.
  • no_db bool — Don’t use the database. Only useful for jobs that initialize things that are required to populate the database.
Attributes
db salvus.flow.db.DB

The currently used database.

full_name str

The job’s full name.

ranks int

Return the maximum number of ranks of all the individual commands.

stderr str

Return the stderr of the job as a string.

stderr_path Union[pathlib.PurePosixPath, pathlib.PureWindowsPath]

The stderr path of the job.

stdout str

Return the stdout of the job as a string.

stdout_path Union[pathlib.PurePosixPath, pathlib.PureWindowsPath]

The stdout path of the job.

Methods
cancel()
def cancel(self, verbosity: int = 1) -> None:
    ...

Cancel job.

Parameters
  • verbosity int — Verbosity level.
Returns None
create_random_job_name()
def create_random_job_name(prefix: str = "job") -> str:
    ...

Create a random name.

Parameters
  • prefix str — Prefix to give some structure to the randomness.
Returns str
delete()
def delete(
    self, verbosity: int = 1, allow_directory_deletion_failure: bool = False
) -> None:
    ...

Deletes the data of a job if it is not running or pending.

Will delete all remote files and the internal database entry.

Parameters
  • verbosity int — Verbosity level.
  • allow_directory_deletion_failure bool — If True this function will still return successfully (but raise a warning) if the deletion of the remote Salvus managed files failed for some reason. This can easily happen on for example networked file systems.
Returns None
get_status_from_db()
def get_status_from_db(self) -> salvus.flow.sites.types.JobStatus:
    ...

Returns the current status of the Job from the database. Please not that this might not be up-to-date.

Returns salvus.flow.sites.types.JobStatus
launch()
def launch(self, verbosity: int = 1, interactive_tty: bool = False) -> None:
    ...

Launch the job.

Parameters
  • verbosity int — Verbosity level.
  • interactive_tty bool — Launch the job in an interactive terminal. Useful for debugging purposes. Only available for local sites.
Returns None
set_status_in_db()
def set_status_in_db(self, status: salvus.flow.sites.types.JobStatus) -> None:
    ...

Set the status in SalvusFlow’s database.

Parameters
  • status salvus.flow.sites.types.JobStatus — The status to be set.
Returns None
tail_stdout()
def tail_stdout(self) -> None:
    ...

tail -f the stdout of the job.

This will launch a remote tail command and it only suitable for interactive use.

Returns None
update_status()
def update_status(
    self,
    progress_callback: Optional[Callable[[Dict], NoneType]] = None,
    force_update: bool = False,
) -> salvus.flow.sites.types.JobStatus:
    ...

Update the status of the job by interacting with the remote site.

Parameters
  • progress_callback Optional[Callable[[Dict], NoneType]] — Callback function which will receive some information about the update.
  • force_update bool — Jobs that are not running and not pending will by default only query the local job database and not check the remote site. In most cases this is correct as the jobs are done with and will not change again. Setting this to True will force an update nonetheless. Uses for this are for example if a job update fails for example due to a temporary network problem.
Returns salvus.flow.sites.types.JobStatus
wait()
def wait(
    self,
    poll_interval_in_seconds: Optional[
        int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64
    ] = None,
    timeout_in_seconds: Optional[float] = None,
    verbosity: int = 1,
    progress_callback: Optional[Callable[[Union[Dict, str]], NoneType]] = None,
) -> salvus.flow.sites.types.JobStatus:
    ...

Wait for the job to finish.

Parameters
  • poll_interval_in_seconds Optional[int, numpy.int32, numpy.int64, float, numpy.float32, numpy.float64] — If given, the site will be queried with this interval. Defaults to the site’s default.
  • timeout_in_seconds Optional[float] — Time in seconds after which this function will return, independent of the status.
  • verbosity int — The verbosity level.
  • progress_callback Optional[Callable[[Union[Dict, str]], NoneType]] — Optional progress callback function. Will be called if there are any status updates. Will currently only be called with the Salvus status file contents.
Returns salvus.flow.sites.types.JobStatus