bilby_pipe.htcondor_sync
bilby_pipe_htcondor_sync can be used to sync jobs running under HTCondor where transfer-files=True (i.e. they do not use a virtual file system). It will use rsync to copy the results from remote worker nodes or spool back to the execute node. The executable is assumed to be run from the submit directory (i.e., where the .ini file is) and on the submit machine (i.e. where the job was submitted from).
Attributes
Functions
|
Get a list of cluster IDs for every running analysis job |
|
Read a log files to determine the latest cluster ID |
|
Attempt to rsync the local (submit) directory to current running worker nodes |
|
Attempt to rsync the local (submit) directory to the spool |
|
Module Contents
- bilby_pipe.htcondor_sync.get_cluster_id_list(outdir)[source]
Get a list of cluster IDs for every running analysis job
- bilby_pipe.htcondor_sync.get_cluster_id(logfile)[source]
Read a log files to determine the latest cluster ID
Extract the HTCondor cluster ID from the .log file. For example, if the log file reads
` 001 (100503183.000.000) 2022-03-07 15:22:49 Job executing on host: <10.14.4.164...> `
Then this function return the cluster ID 100503183- Parameters:
- logfile: str
A path to a HTCondor log file
- Returns:
- cluster_id: str
The cluster ID. If not ID is found, None is returned and a log message is printed.
- bilby_pipe.htcondor_sync.rsync_via_ssh(cluster_id, outdir, verbose=False, timeout=30)[source]
Attempt to rsync the local (submit) directory to current running worker nodes
This method applies when the job is actively executing on a remote worker node and condor_ssh_to_job is possible. The method works by using condor_ssh_to_job and rsync as described in the HTCondor documentation: https://htcondor.readthedocs.io/en/latest/man-pages/condor_ssh_to_job.html.
- Parameters:
- cluster_id: int
The HTCondor clusterId
- outdir: str
The top-level outdir of the bilby_pipe job
- verbose: bool
If true, print explicit error messages
- timeout: int
The timeout interval
- Returns:
- success: bool
True if the method was successful.
- bilby_pipe.htcondor_sync.rsync_via_spool(cluster_id, outdir, verbose=False)[source]
Attempt to rsync the local (submit) directory to the spool
This method applies when the job is not actively executing on a remote worker, but is idle. In this instance, any files produced by the job will be stored in the spool (a local directory on the submit machine). This methods identifies the spool location, based on the cluster_id, and attempts to rsync the data.
- Parameters:
- cluster_id: int
The HTCondor clusterId
- outdir: str
The top-level outdir of the bilby_pipe job
- Returns:
- success: bool
True if the method was successful.