Mondaic Documentation
www.mondaic.com
Mondaic
Version:
No results found.
python_api
/
salvus
/
flow
/
executors
/
salvus_job

salvus.flow.executors.salvus_job

salvus.flow.executors.salvus_job salvus flow executors salvus_job
A job class specific to running SalvusCompute.

Classes

SalvusJob

A class representing a concrete salvus job associated with a certain site.
All interactions like updating the job or persisting some things to the database should flow over this class.
SIGNATURE
class SalvusJob(salvus.flow.executors.job.Job):
    def __init__(
        self,
        site: BaseExecutor,
        input_file: str | pathlib.Path | dict | utils.deep_setters._DeepSetter,
        remote_input_file: (
            str
            | pathlib.PurePosixPath
            | dict
            | utils.deep_setters._DeepSetter
            | None
        ) = None,
        ranks: salvus._core.types.int_ | None = None,
        wall_time_in_seconds: salvus._core.types.float_ | None = None,
        job_groups: list[db.JobGroup | str] | None = None,
        is_debug_job: bool = False,
        verbosity: int = 1,
        jobname: str | None = None,
        initialize_on_site: bool = True,
        rundir_root: _RemotePathInputType | None = None,
        tmpdir_root: _RemotePathInputType | None = None,
        working_dir: _RemotePathInputType | None = None,
        skip_schema_validation: bool = False,
        db_command_line_call: str | None = None,
        allow_uninitialized_site: bool = False,
        precomputed_remote_file_hashes: dict | None = None,
        parsed_input_file: (
            tuple[dict, dict, dict, None | pathlib.Path] | None
        ) = None,
    ): ...
ARGUMENTS
Required
site
Type:BaseExecutor
Description:
The site this job is to be run at.
Required
input_file
Type:str | pathlib.Path | dict | utils.deep_setters._DeepSetter
Description:
Salvus input file.
Optional
remote_input_file
Type:str | pathlib.PurePosixPath | dict | utils.deep_setters._DeepSetter | None
Default value:None
Description:
The input file as seen by the remote site.
Optional
ranks
Type:salvus._core.types.int_ | None
Default value:None
Description:
Number of ranks to use.
Optional
wall_time_in_seconds
Type:salvus._core.types.float_ | None
Default value:None
Description:
Wall time in seconds for the job. Not required for jobs on all sites.
Optional
job_groups
Type:list[db.JobGroup | str] | None
Default value:None
Description:
List of job groups names this job should be part of.
Optional
is_debug_job
Type:bool
Default value:False
Description:
True if this job should be executed in the fast/debug queue if the site has one.
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Optional
jobname
Type:str | None
Default value:None
Description:
Name of the job. A random one will be created if none is given.
Optional
initialize_on_site
Type:bool
Default value:True
Description:
Initialize on site. Should be true in most cases. False is useful for creating a Job object from the database.
Optional
rundir_root
Type:_RemotePathInputType | None
Default value:None
Description:
Specify what parent folder the run directory will be created in. Defaults to the directory in the site config.
Optional
tmpdir_root
Type:_RemotePathInputType | None
Default value:None
Description:
Specify what parent folder the tmp directory will be created in. Defaults to the directory in the site config.
Optional
working_dir
Type:_RemotePathInputType | None
Default value:None
Description:
Working directory. If not given it will be set to the job directory. Should only be set if the job is part of a job array.
Optional
skip_schema_validation
Type:bool
Default value:False
Description:
Optionally skip the schema validation. Only use when reading jobs from the database and for site initialization.
Optional
db_command_line_call
Type:str | None
Default value:None
Description:
Command line call of this job for the database.
Optional
allow_uninitialized_site
Type:bool
Default value:False
Description:
Allows an uninitialized site - intended to only be used for the job that initializes the site.
Optional
precomputed_remote_file_hashes
Type:dict | None
Default value:None
Description:
List of potentially precomputed hashes for remote files. If given they do not need to be computed for files on the remote site.
Optional
parsed_input_file
Type:tuple[dict, dict, dict, None | pathlib.Path] | None
Default value:None
Description:
Can pass an already parsed input file (parsed with site_utils.parse_salvus_input) so it does not happen again. Probably only useful for job arrays.

Properties

  • db(salvus_flow_db.DB)-The currently used database.
  • full_name(str)-Returns the full name of a job as JOB_NAME@SITE_NAME.
  • meta_json_path(_RemotePathType)-Path of the job's meta JSON file.
  • ranks(int)-Return the maximum number of ranks of all the individual commands.
  • salvus_progress_file_path(_RemotePathType)-Path of the job's progress JSON file.
  • stderr(str)-Return the stderr of the job as a string.
  • stderr_path(_RemotePathType)-The stderr path of the job.
  • stdout(str)-Return the stdout of the job as a string.
  • stdout_path(_RemotePathType)-The stdout path of the job.

Methods

SalvusJob.cancel()
Cancel job.
SIGNATURE
def cancel(self, verbosity: int = 1) -> None: ...
ARGUMENTS
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
SalvusJob.copy_output()
Copy the output of a finished job to a target directory.
SIGNATURE
def copy_output(
    self,
    destination: pathlib.Path | str,
    get_all: bool = False,
    verbosity: int = 1,
    allow_existing_destination_folder: bool = False,
) -> tuple[dict[pathlib.Path, int], bool]: ...
ARGUMENTS
Required
destination
Type:pathlib.Path | str
Description:
The destination folder. Must not yet exist.
Optional
get_all
Type:bool
Default value:False
Description:
Also get the large output files, if any.
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Optional
allow_existing_destination_folder
Type:bool
Default value:False
Description:
Allow that the destination already exists. Files might be overwritten.
SalvusJob.create_random_job_name()
Create a random name.
SIGNATURE
def create_random_job_name(prefix: str = "job") -> str: ...
ARGUMENTS
Optional
prefix
Type:str
Default value:'job'
Description:
Prefix to give some structure to the randomness.
SalvusJob.delete()
Delete all data for a given job.
Will delete all remote files and the internal database entry.
SIGNATURE
def delete(
    self, verbosity: int = 1, allow_directory_deletion_failure: bool = False
) -> None: ...
ARGUMENTS
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Optional
allow_directory_deletion_failure
Type:bool
Default value:False
Description:
If True this function will still return successfully (but raise a warning) if the deletion of the remote Salvus managed files failed for some reason. This can easily happen on for example networked file systems.
SalvusJob.get_as_event()
Return the receivers of this simulation as an EventData object.
Can only be called if the output data of this job has been downloaded to the local machine.
SIGNATURE
def get_as_event(self) -> EventData: ...
SalvusJob.get_current_progress()
Gets the contents of the contents of the Salvus progress JSON file.
Returns None if it could not find the file.
SIGNATURE
def get_current_progress(self) -> dict | None: ...
SalvusJob.get_diagnosis_string()
Get a diagnostic string with the stdout/stderr from the job.
Useful for trying to figure out why a job fails.
SIGNATURE
def get_diagnosis_string(self) -> str: ...
SalvusJob.get_output_files()
Get a dictionary containing the output files.
The keys are either "stderr", "stdout", or a tuple denoting the path of the file in the input file. The values the full paths. This does parse the input.toml file to figure out what is there.
>>> j.get_output_files()
{'stderr': PurePosixPath('run/job_1905231521_8d3f5/stderr'),
 'stdout': PurePosixPath('run/job_1905231521_8d3f5/stdout'),
 ('output', 'meta_data', 'meta_json_filename'):
    PurePosixPath('run/job_1905231521_8d3f5/output/meta.json')}
SIGNATURE
def get_output_files(
    self, get_all: bool = False
) -> tuple[dict[str | tuple, _RemotePathType], bool]: ...
ARGUMENTS
Optional
get_all
Type:bool
Default value:False
Description:
Also get the all outputs designated as large. This will usually be volumetric and/or boundary output.
SalvusJob.get_status_from_db()
Returns the current status of the Job from the database. Please not that this might not be up-to-date.
SIGNATURE
def get_status_from_db(self) -> JobStatus: ...
SalvusJob.launch()
Launch the job.
SIGNATURE
def launch(
    self, verbosity: int = 1, interactive_tty: bool = False
) -> None: ...
ARGUMENTS
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Optional
interactive_tty
Type:bool
Default value:False
Description:
Launch the job in an interactive terminal. Useful for debugging purposes. Only available for local sites.
SalvusJob.set_status_in_db()
Set the status in SalvusFlow's database.
SIGNATURE
def set_status_in_db(self, status: JobStatus) -> None: ...
ARGUMENTS
Required
status
Type:JobStatus
Description:
The status to be set.
SalvusJob.tail_stdout()
tail -f the stdout of the job.
This will launch a remote tail command and it only suitable for interactive use.
SIGNATURE
def tail_stdout(self) -> None: ...
SalvusJob.update_status()
Updates the status of a given job and persist it in the database.
(1) It will first try to find and parse the progress.json file that is written to during Salvus runs. This is retried up to 20 times in case of some spurious failure like a network issue or by chance parsing it right when it is being written by Salvus.
(2a) If the progress file does not exist: fall back to the parent class' logic and get the status of the process: * If not finished: accept and return. * If finished: parse meta.json, persist in DB, and return.
(2b) If the progress file exists: * If status != "finished": Consider it to be running, persist and return. * If status == "finished": Parse meta.json file, persist in DB and return.
Note that the last part of both branches is identical.
Unfortunately a somewhat complex function but it should be fairly resilient against race conditions.
SIGNATURE
def update_status(
    self,
    progress_callback: collections.abc.Callable[[dict], None] | None = None,
    force_update: bool = False,
) -> JobStatus: ...
ARGUMENTS
Optional
progress_callback
Type:collections.abc.Callable[[dict], None] | None
Default value:None
Description:
Callback function that is called whenever something happens.
Optional
force_update
Type:bool
Default value:False
Description:
Jobs that are not running and not pending will by default only query the local job database and not check the remote site. In most cases this is correct as the jobs are done with and will not change again. Setting this to True will force an update nonetheless. Uses for this are for example if a job update fails for example due to a temporary network problem.
SalvusJob.wait()
Wait for the job to finish.
SIGNATURE
def wait(
    self,
    poll_interval_in_seconds: salvus._core.types.float_ | None = None,
    timeout_in_seconds: float | None = None,
    verbosity: int = 1,
    progress_callback: (
        collections.abc.Callable[[dict | str], None] | None
    ) = None,
) -> JobStatus: ...
ARGUMENTS
Optional
poll_interval_in_seconds
Type:salvus._core.types.float_ | None
Default value:None
Description:
If given, the site will be queried with this interval. Defaults to the site's default.
Optional
timeout_in_seconds
Type:float | None
Default value:None
Description:
Time in seconds after which this function will return, independent of the status.
Optional
verbosity
Type:int
Default value:1
Description:
The verbosity level.
Optional
progress_callback
Type:collections.abc.Callable[[dict | str], None] | None
Default value:None
Description:
Optional progress callback function. Will be called if there are any status updates. Will currently only be called with the Salvus status file contents.

Used in tutorials

Functionality from this module is used in the following tutorials.
Advanced Interface / Manual Gradients / Manual Gradient Computation
Advanced Interface / Salvus Flow Api / Introduction to SalvusFlow's API
Previous
remote_io_executor
Next
salvus_job_array
We use cookies to analyze our traffic, but won't track you until explicit consent is given. Using our contact form will create relevant cookies and transmit data to third party services to keep track of your request. Read more about how we use personal information in our Privacy Policy.
PAGE CONTENTS