Mondaic Documentation
www.mondaic.com
Mondaic
Version:
No results found.
python_api
/
salvus
/
flow
/
executors
/
implementations
/
ssh

salvus.flow.executors.implementations.ssh

salvus.flow.executors.implementations.ssh salvus flow executors implementations ssh
An executor for SSH connected sites.

Classes

SSHExecutor

Site that will directly execute a job over SSH.
It attempts to be UNIX and Windows compatible.
There are a few assumptions about MPI.
On Unix it must be installed relative to the Salvus binary:
/bin
    mpirun
    salvus
/lib
    mpi_libs
On Windows MS-MPI must be installed and available via mpiexec.exe.
SIGNATURE
class SSHExecutor(salvus.flow.executors.base_executor.BaseExecutor):
    def __init__(
        self,
        name: str,
        configuration: dict,
        db: typing.Any,
        verbosity: int = 1,
        skip_version_number_check: bool = False,
    ): ...
ARGUMENTS
Required
name
Type:str
Description:
Name of the site using the executor.
Required
configuration
Type:dict
Description:
Configuration for the site.
Required
db
Type:typing.Any
Description:
Connection to the internal database.
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Optional
skip_version_number_check
Type:bool
Default value:False
Description:
Skip the version number check of the local Python version vs the remote site. Useful for initializing and updating sites.

Properties

  • file_path_type(type[pathlib.PurePosixPath | pathlib.PureWindowsPath])-Returns the type of file paths for the given site.
  • home_dir(pathlib.PurePosixPath | pathlib.PureWindowsPath)-The home directory.
  • is_initialized(bool)-True if the site has been synchronized/initialized with the remote site.
  • os_type(OSType)-Returns the type of the operating system. Subclasses can overwrite this.
  • sftp_client(paramiko.sftp_client.SFTPClient | LocalSFTPClient)-Get the sftp client instance. Will be initialized upon the first access.
  • ssh_client(paramiko.client.SSHClient | LocalSSHClient)-Get the ssh client instance. Will be initialized upon the first access.
  • update_interval_in_seconds(float)-A site's actual update interval.
  • use_cuda_capable_gpus(bool)-True if a site uses license tokens.
  • use_license_tokens(float)-True if a site uses license tokens.

Methods

SSHExecutor.cancel_job()
Cancel the job.
SIGNATURE
def cancel_job(self, job_dir: str) -> None: ...
ARGUMENTS
Required
job_dir
Type:str
Description:
The job directory.
SSHExecutor.execute_command()
Execute the command.
SIGNATURE
def execute_command(
    self,
    command: str,
    assert_ok: bool = True,
    environment: dict[str, str] | None = None,
) -> tuple[int, list[str], list[str]]: ...
ARGUMENTS
Required
command
Type:str
Description:
The command to run as a string.
Optional
assert_ok
Type:bool
Default value:True
Description:
Raise if the command does not exist with a code of 0.
Optional
environment
Type:dict[str, str] | None
Default value:None
Description:
Environment variables to be set for this command.
SSHExecutor.get_job_status()
Return the status of the job or job array to the best knowledge of the remote site.
If the job is a job array, return the status of every job in a list if possible, otherwise just the total status of the whole job.
This is a single method as, in practice, it turns out to be more convenient to implement.
SIGNATURE
def get_job_status(self, job_dir: str) -> JobStatus: ...
ARGUMENTS
Required
job_dir
Type:str
Description:
The job directory.
SSHExecutor.is_folder_writeable()
Return True if the user has write permissions in the chosen folder.
SIGNATURE
def is_folder_writeable(self, path: _RemotePathInputType) -> bool: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Path on the remote machine.
SSHExecutor.launch_many_mpi_jobs()
Launch many MPI jobs at once.
SIGNATURE
def launch_many_mpi_jobs(
    self,
    job_commands: dict[int, typing.Any],
    working_dir: str,
    stdout_path: str,
    stderr_path: str,
    wall_time_in_seconds_per_job: int | None,
    license_tokens: list[str] | None = None,
) -> list[JobStatus]: ...
ARGUMENTS
Required
job_commands
Type:dict[int, typing.Any]
Description:
Dictionary of a list of commands, one per list per job.
Required
working_dir
Type:str
Description:
The remote working directory for the execution.
Required
stdout_path
Type:str
Description:
Path to pipe stdout to.
Required
stderr_path
Type:str
Description:
Path to pipe stderr to.
Required
wall_time_in_seconds_per_job
Type:int | None
Description:
Wall-time in seconds per job. Not necessary for the SSH site.
Optional
license_tokens
Type:list[str] | None
Default value:None
Description:
If the job array uses license tokens for the license enforcement they are passed here.
SSHExecutor.launch_mpi_job()
Run an MPI job.
Must return the initial status. Should either be "running" or "pending", depending on the system.
SIGNATURE
def launch_mpi_job(
    self,
    commands: list[executor_utils.RemoteCommand],
    job_dir: str,
    working_dir: str,
    stdout_path: str,
    stderr_path: str,
    wall_time_in_seconds: int | None,
    is_debug_job: bool = False,
    interactive_tty: bool = False,
) -> JobStatus: ...
ARGUMENTS
Required
commands
Type:list[executor_utils.RemoteCommand]
Description:
List of commands. One will be run after the other, but all will be submitted at the same time.
Required
job_dir
Type:str
Description:
The remote directory where all the job files are stored.
Required
working_dir
Type:str
Description:
The remote working directory for the execution.
Required
stdout_path
Type:str
Description:
Path to pipe stdout to.
Required
stderr_path
Type:str
Description:
Path to pipe stderr to.
Required
wall_time_in_seconds
Type:int | None
Description:
Wall-time in seconds. Might be None, depending on the site.
Optional
is_debug_job
Type:bool
Default value:False
Description:
True if this is supposed to be a debug job. Some job management systems have debug or other "fast" queues. Use these special queues if this is set to true.
Optional
interactive_tty
Type:bool
Default value:False
Description:
Launch the job in an interactive terminal. This only works for local sites, which are based on this SSH site.
SSHExecutor.pretty_print()
Pretty and colorful representation of the site.
SIGNATURE
def pretty_print(self) -> None: ...
SSHExecutor.remote_exists()
Check if a remote file or directory exists.
SIGNATURE
def remote_exists(self, path: _RemotePathInputType) -> bool: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Path to check.
SSHExecutor.remote_get()
Get a remote file.
SIGNATURE
def remote_get(
    self,
    remotepath: _RemotePathInputType,
    localpath: io.BytesIO | pathlib.Path,
    show_progressbar: bool = False,
) -> None: ...
ARGUMENTS
Required
remotepath
Type:_RemotePathInputType
Description:
Remote file to get.
Required
localpath
Type:io.BytesIO | pathlib.Path
Description:
Local path to save to Can be a file or a file like object.
Optional
show_progressbar
Type:bool
Default value:False
Description:
Show a progressbar or not.
SSHExecutor.remote_listdir()
List the contents of a remote directory.
Just dispatches to the underlying representation.
SIGNATURE
def remote_listdir(self, path: _RemotePathInputType) -> list[str]: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Remote path whose contents to list.
SSHExecutor.remote_mkdir()
Create a directory on the remote or local machine.
SIGNATURE
def remote_mkdir(
    self, path: _RemotePathInputType, mode: int = 511
) -> None: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Directory to create.
Optional
mode
Type:int
Default value:511
Description:
Permissions for the new directory.
SSHExecutor.remote_put()
Copy a local file to a remote path.
SIGNATURE
def remote_put(
    self,
    localpath: io.BytesIO | pathlib.Path,
    remotepath: _RemotePathInputType,
    show_progressbar: bool = False,
) -> None: ...
ARGUMENTS
Required
localpath
Type:io.BytesIO | pathlib.Path
Description:
Local file to copy. Can be a file like object.
Required
remotepath
Type:_RemotePathInputType
Description:
Where to copy to.
Optional
show_progressbar
Type:bool
Default value:False
Description:
Show a progressbar or not.
SSHExecutor.remote_rmdir()
Remotely and recursively delete a remote folder and its contents.
SIGNATURE
def remote_rmdir(
    self, path: _RemotePathInputType, verbosity: int = 1
) -> None: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Remote path to delete.
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
SSHExecutor.safe_remote_rmdir()
Convenience wrapper that optionally allows the remote_rmdir() command to fail. Useful for some networked file systems.
SIGNATURE
def safe_remote_rmdir(
    self, path: _RemotePathInputType, allow_failure: bool, verbosity: int = 1
) -> None: ...
ARGUMENTS
Required
path
Type:_RemotePathInputType
Description:
Remote path to delete.
Required
allow_failure
Type:bool
Description:
Whether to allow failure or not.
Optional
verbosity
Type:int
Default value:1
Description:
Verbosity level.
Previous
slurm
Next
local_hpc
We use cookies to analyze our traffic, but won't track you until explicit consent is given. Using our contact form will create relevant cookies and transmit data to third party services to keep track of your request. Read more about how we use personal information in our Privacy Policy.
PAGE CONTENTS