Detailed Description

A CondorDAGNode represents a node in the DAG.

It corresponds to a particular condor job (and so a particular submit file). If the job has variable (macro) options, they can be set here so each nodes executes with the correct options.

Definition at line 647 of file pipeline.py.

Inherits object.

Inherited by lal.pipeline.CondorDAGManNode, lal.pipeline.LSCDataFindNode, lal.pipeline.LigolwAddNode, lal.pipeline.LigolwCutNode, lal.pipeline.NoopNode, and lal.pipeline.SqliteNode.

Public Member Functions
def	__init__ (self, job)

def	__repr__ (self)

def	job (self)
	Return the CondorJob that this node is associated with. More...

def	set_pre_script (self, script)
	Sets the name of the pre script that is executed before the DAG node is run. More...

def	add_pre_script_arg (self, arg)
	Adds an argument to the pre script that is executed before the DAG node is run. More...

def	set_post_script (self, script)
	Sets the name of the post script that is executed before the DAG node is run. More...

def	get_post_script (self)
	returns the name of the post script that is executed before the DAG node is run. More...

def	add_post_script_arg (self, arg)
	Adds an argument to the post script that is executed before the DAG node is run. More...

def	get_post_script_arg (self)
	Returns and array of arguments to the post script that is executed before the DAG node is run. More...

def	set_name (self, name)
	Set the name for this node in the DAG. More...

def	get_name (self)
	Get the name for this node in the DAG. More...

def	set_category (self, category)
	Set the category for this node in the DAG. More...

def	get_category (self)
	Get the category for this node in the DAG. More...

def	set_priority (self, priority)
	Set the priority for this node in the DAG. More...

def	get_priority (self)
	Get the priority for this node in the DAG. More...

def	add_input_file (self, filename)
	Add filename as a necessary input file for this DAG node. More...

def	add_output_file (self, filename)
	Add filename as a output file for this DAG node. More...

def	add_checkpoint_file (self, filename)
	Add filename as a checkpoint file for this DAG node. More...

def	get_input_files (self)
	Return list of input files for this DAG node and its job. More...

def	get_output_files (self)
	Return list of output files for this DAG node and its job. More...

def	get_checkpoint_files (self)
	Return a list of checkpoint files for this DAG node and its job. More...

def	set_vds_group (self, group)
	Set the name of the VDS group key when generating a DAX. More...

def	get_vds_group (self)
	Returns the VDS group key for this node. More...

def	add_macro (self, name, value)
	Add a variable (macro) for this node. More...

def	add_io_macro (self, io, filename)
	Add a variable (macro) for storing the input/output files associated with this node. More...

def	add_input_macro (self, filename)
	Add a variable (macro) for storing the input files associated with this node. More...

def	add_output_macro (self, filename)
	Add a variable (macro) for storing the output files associated with this node. More...

def	add_checkpoint_macro (self, filename)

def	get_opts (self)
	Return the opts for this node. More...

def	add_var_condor_cmd (self, command, value)
	Add a variable (macro) condor command for this node. More...

def	add_var_opt (self, opt, value, short=False)
	Add a variable (macro) option for this node. More...

def	add_file_opt (self, opt, filename, file_is_output_file=False)
	Add a variable (macro) option for this node. More...

def	add_var_arg (self, arg, quote=False)
	Add a variable (or macro) argument to the condor job. More...

def	add_file_arg (self, filename)
	Add a variable (or macro) file name argument to the condor job. More...

def	get_args (self)
	Return the arguments for this node. More...

def	set_retry (self, retry)
	Set the number of times that this node in the DAG should retry. More...

def	get_retry (self)
	Return the number of times that this node in the DAG should retry. More...

def	write_job (self, fh)
	Write the DAG entry for this node's job to the DAG file descriptor. More...

def	write_category (self, fh)
	Write the DAG entry for this node's category to the DAG file descriptor. More...

def	write_priority (self, fh)
	Write the DAG entry for this node's priority to the DAG file descriptor. More...

def	write_vars (self, fh)
	Write the variable (macro) options and arguments to the DAG file descriptor. More...

def	write_parents (self, fh)
	Write the parent/child relations for this job to the DAG file descriptor. More...

def	write_pre_script (self, fh)
	Write the pre script for the job, if there is one. More...

def	write_post_script (self, fh)
	Write the post script for the job, if there is one. More...

def	write_input_files (self, fh)
	Write as a comment into the DAG file the list of input files for this DAG node. More...

def	write_output_files (self, fh)
	Write as a comment into the DAG file the list of output files for this DAG node. More...

def	set_log_file (self, log)
	Set the Condor log file to be used by this CondorJob. More...

def	add_parent (self, node)
	Add a parent to this node. More...

def	get_cmd_tuple_list (self)
	Return a list of tuples containg the command line arguments. More...

def	get_cmd_line (self)
	Return the full command line that will be used when this node is run by DAGman. More...

def	finalize (self)
	The finalize method of a node is called before the node is finally added to the DAG and can be overridden to do any last minute clean up (such as setting extra command line arguments) More...

Constructor & Destructor Documentation

◆ init()

def lal.pipeline.CondorDAGNode.__init__	(	self,
		job
	)

Parameters

job	the CondorJob that this node corresponds to.

Reimplemented in lal.pipeline.CondorDAGManNode, lal.pipeline.LSCDataFindNode, lal.pipeline.LigolwAddNode, lal.pipeline.LigolwCutNode, lal.pipeline.NoopNode, lal.pipeline.SqliteNode, and lal.pipeline.LigolwSqliteNode.

Definition at line 651 of file pipeline.py.

Member Function Documentation

◆ repr()

def lal.pipeline.CondorDAGNode.__repr__ ( self )

Definition at line 687 of file pipeline.py.

◆ job()

def lal.pipeline.CondorDAGNode.job ( self )

Return the CondorJob that this node is associated with.

Definition at line 693 of file pipeline.py.

◆ set_pre_script()

def lal.pipeline.CondorDAGNode.set_pre_script	(	self,
		script
	)

Sets the name of the pre script that is executed before the DAG node is run.

Parameters

script path to script

Definition at line 701 of file pipeline.py.

◆ add_pre_script_arg()

def lal.pipeline.CondorDAGNode.add_pre_script_arg	(	self,
		arg
	)

Adds an argument to the pre script that is executed before the DAG node is run.

Definition at line 708 of file pipeline.py.

◆ set_post_script()

def lal.pipeline.CondorDAGNode.set_post_script	(	self,
		script
	)

Sets the name of the post script that is executed before the DAG node is run.

Parameters

script path to script

Definition at line 716 of file pipeline.py.

◆ get_post_script()

def lal.pipeline.CondorDAGNode.get_post_script ( self )

returns the name of the post script that is executed before the DAG node is run.

Definition at line 723 of file pipeline.py.

◆ add_post_script_arg()

def lal.pipeline.CondorDAGNode.add_post_script_arg	(	self,
		arg
	)

Adds an argument to the post script that is executed before the DAG node is run.

Definition at line 730 of file pipeline.py.

◆ get_post_script_arg()

def lal.pipeline.CondorDAGNode.get_post_script_arg ( self )

Returns and array of arguments to the post script that is executed before the DAG node is run.

Definition at line 737 of file pipeline.py.

◆ set_name()

def lal.pipeline.CondorDAGNode.set_name	(	self,
		name
	)

Set the name for this node in the DAG.

Definition at line 743 of file pipeline.py.

◆ get_name()

def lal.pipeline.CondorDAGNode.get_name ( self )

Get the name for this node in the DAG.

Definition at line 749 of file pipeline.py.

◆ set_category()

def lal.pipeline.CondorDAGNode.set_category	(	self,
		category
	)

Set the category for this node in the DAG.

Definition at line 755 of file pipeline.py.

◆ get_category()

def lal.pipeline.CondorDAGNode.get_category ( self )

Get the category for this node in the DAG.

Definition at line 761 of file pipeline.py.

◆ set_priority()

def lal.pipeline.CondorDAGNode.set_priority	(	self,
		priority
	)

Set the priority for this node in the DAG.

Definition at line 767 of file pipeline.py.

◆ get_priority()

def lal.pipeline.CondorDAGNode.get_priority ( self )

Get the priority for this node in the DAG.

Definition at line 773 of file pipeline.py.

◆ add_input_file()

def lal.pipeline.CondorDAGNode.add_input_file	(	self,
		filename
	)

Add filename as a necessary input file for this DAG node.

Parameters

filename input filename to add

Definition at line 781 of file pipeline.py.

◆ add_output_file()

def lal.pipeline.CondorDAGNode.add_output_file	(	self,
		filename
	)

Add filename as a output file for this DAG node.

Parameters

filename output filename to add

Definition at line 793 of file pipeline.py.

◆ add_checkpoint_file()

def lal.pipeline.CondorDAGNode.add_checkpoint_file	(	self,
		filename
	)

Add filename as a checkpoint file for this DAG node.

Parameters

filename checkpoint filename to add

Definition at line 804 of file pipeline.py.

◆ get_input_files()

def lal.pipeline.CondorDAGNode.get_input_files ( self )

Return list of input files for this DAG node and its job.

Definition at line 814 of file pipeline.py.

◆ get_output_files()

def lal.pipeline.CondorDAGNode.get_output_files ( self )

Return list of output files for this DAG node and its job.

Definition at line 823 of file pipeline.py.

◆ get_checkpoint_files()

def lal.pipeline.CondorDAGNode.get_checkpoint_files ( self )

Return a list of checkpoint files for this DAG node and its job.

Definition at line 832 of file pipeline.py.

◆ set_vds_group()

def lal.pipeline.CondorDAGNode.set_vds_group	(	self,
		group
	)

Set the name of the VDS group key when generating a DAX.

Parameters

group name of group for thus nore

Definition at line 842 of file pipeline.py.

◆ get_vds_group()

def lal.pipeline.CondorDAGNode.get_vds_group ( self )

Returns the VDS group key for this node.

Definition at line 848 of file pipeline.py.

◆ add_macro()

def lal.pipeline.CondorDAGNode.add_macro	(	self,
		name,
		value
	)

Add a variable (macro) for this node.

This can be different for each node in the DAG, even if they use the same CondorJob. Within the CondorJob, the value of the macro can be referenced as '' – for instance, to define a unique output or error file for each node.

Parameters

name	macro name.
value	value of the macro for this node in the DAG

Definition at line 860 of file pipeline.py.

◆ add_io_macro()

def lal.pipeline.CondorDAGNode.add_io_macro	(	self,
		io,
		filename
	)

Add a variable (macro) for storing the input/output files associated with this node.

Parameters

io	macroinput or macrooutput
filename	filename of input/output file

Definition at line 870 of file pipeline.py.

◆ add_input_macro()

def lal.pipeline.CondorDAGNode.add_input_macro	(	self,
		filename
	)

Add a variable (macro) for storing the input files associated with this node.

Parameters

filename filename of input file

Definition at line 883 of file pipeline.py.

◆ add_output_macro()

def lal.pipeline.CondorDAGNode.add_output_macro	(	self,
		filename
	)

Add a variable (macro) for storing the output files associated with this node.

Parameters

filename filename of output file

Definition at line 891 of file pipeline.py.

◆ add_checkpoint_macro()

def lal.pipeline.CondorDAGNode.add_checkpoint_macro	(	self,
		filename
	)

Definition at line 894 of file pipeline.py.

◆ get_opts()

def lal.pipeline.CondorDAGNode.get_opts ( self )

Return the opts for this node.

Note that this returns only the options for this instance of the node and not those associated with the underlying job template.

Definition at line 902 of file pipeline.py.

◆ add_var_condor_cmd()

def lal.pipeline.CondorDAGNode.add_var_condor_cmd	(	self,
		command,
		value
	)

Add a variable (macro) condor command for this node.

If the command specified does not exist in the CondorJob, it is added so the submit file will be correct. PLEASE NOTE: AS with other add_var commands, the variable must be set for all nodes that use the CondorJob instance.

Parameters

command	command name
value	Value of the command for this node in the DAG/DAX.

Definition at line 914 of file pipeline.py.

◆ add_var_opt()

def lal.pipeline.CondorDAGNode.add_var_opt	(	self,
		opt,
		value,
		short = `False`
	)

Add a variable (macro) option for this node.

If the option specified does not exist in the CondorJob, it is added so the submit file will be correct when written.

Parameters

opt	option name.
value	value of the option for this node in the DAG.
short	opt is short

Definition at line 927 of file pipeline.py.

◆ add_file_opt()

def lal.pipeline.CondorDAGNode.add_file_opt	(	self,
		opt,
		filename,
		file_is_output_file = `False`
	)

Add a variable (macro) option for this node.

If the option specified does not exist in the CondorJob, it is added so the submit file will be correct when written. The value of the option is also added to the list of input files for the DAX.

Parameters

opt	option name.
filename	value of the option for this node in the DAG.
file_is_output_file	A boolean if the file will be an output file instead of an input file. The default is to have it be an input.

Definition at line 942 of file pipeline.py.

◆ add_var_arg()

def lal.pipeline.CondorDAGNode.add_var_arg	(	self,
		arg,
		quote = `False`
	)

Add a variable (or macro) argument to the condor job.

The argument is added to the submit file and a different value of the argument can be set for each node in the DAG.

Parameters

arg	name of option to add.
quote	quote

Definition at line 954 of file pipeline.py.

◆ add_file_arg()

def lal.pipeline.CondorDAGNode.add_file_arg	(	self,
		filename
	)

Add a variable (or macro) file name argument to the condor job.

The argument is added to the submit file and a different value of the argument can be set for each node in the DAG. The file name is also added to the list of input files for the DAX.

Parameters

filename name of option to add.

Definition at line 966 of file pipeline.py.

◆ get_args()

def lal.pipeline.CondorDAGNode.get_args ( self )

Return the arguments for this node.

Note that this returns only the arguments for this instance of the node and not those associated with the underlying job template.

Definition at line 975 of file pipeline.py.

◆ set_retry()

def lal.pipeline.CondorDAGNode.set_retry	(	self,
		retry
	)

Set the number of times that this node in the DAG should retry.

Parameters

retry number of times to retry node.

Definition at line 982 of file pipeline.py.

◆ get_retry()

def lal.pipeline.CondorDAGNode.get_retry ( self )

Return the number of times that this node in the DAG should retry.

Definition at line 988 of file pipeline.py.

◆ write_job()

def lal.pipeline.CondorDAGNode.write_job	(	self,
		fh
	)

Write the DAG entry for this node's job to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.

Definition at line 995 of file pipeline.py.

◆ write_category()

def lal.pipeline.CondorDAGNode.write_category	(	self,
		fh
	)

Write the DAG entry for this node's category to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.

Definition at line 1013 of file pipeline.py.

◆ write_priority()

def lal.pipeline.CondorDAGNode.write_priority	(	self,
		fh
	)

Write the DAG entry for this node's priority to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.

Definition at line 1020 of file pipeline.py.

◆ write_vars()

def lal.pipeline.CondorDAGNode.write_vars	(	self,
		fh
	)

Write the variable (macro) options and arguments to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.

Definition at line 1028 of file pipeline.py.

◆ write_parents()

def lal.pipeline.CondorDAGNode.write_parents	(	self,
		fh
	)

Write the parent/child relations for this job to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.

Definition at line 1044 of file pipeline.py.

◆ write_pre_script()

def lal.pipeline.CondorDAGNode.write_pre_script	(	self,
		fh
	)

Write the pre script for the job, if there is one.

Parameters

fh	descriptor of open DAG file.

Definition at line 1052 of file pipeline.py.

◆ write_post_script()

def lal.pipeline.CondorDAGNode.write_post_script	(	self,
		fh
	)

Write the post script for the job, if there is one.

Parameters

fh	descriptor of open DAG file.

Definition at line 1061 of file pipeline.py.

◆ write_input_files()

def lal.pipeline.CondorDAGNode.write_input_files	(	self,
		fh
	)

Write as a comment into the DAG file the list of input files for this DAG node.

Parameters

fh	descriptor of open DAG file.

Definition at line 1072 of file pipeline.py.

◆ write_output_files()

def lal.pipeline.CondorDAGNode.write_output_files	(	self,
		fh
	)

Write as a comment into the DAG file the list of output files for this DAG node.

Parameters

fh	descriptor of open DAG file.

Definition at line 1082 of file pipeline.py.

◆ set_log_file()

def lal.pipeline.CondorDAGNode.set_log_file	(	self,
		log
	)

Set the Condor log file to be used by this CondorJob.

Parameters

log	path of Condor log file.

Definition at line 1090 of file pipeline.py.

◆ add_parent()

def lal.pipeline.CondorDAGNode.add_parent	(	self,
		node
	)

Add a parent to this node.

This node will not be executed until the parent node has run sucessfully.

Parameters

node	CondorDAGNode to add as a parent.

Definition at line 1098 of file pipeline.py.

◆ get_cmd_tuple_list()

def lal.pipeline.CondorDAGNode.get_cmd_tuple_list ( self )

Return a list of tuples containg the command line arguments.

Definition at line 1106 of file pipeline.py.

◆ get_cmd_line()

def lal.pipeline.CondorDAGNode.get_cmd_line ( self )

Return the full command line that will be used when this node is run by DAGman.

Definition at line 1164 of file pipeline.py.

◆ finalize()

def lal.pipeline.CondorDAGNode.finalize ( self )

The finalize method of a node is called before the node is finally added to the DAG and can be overridden to do any last minute clean up (such as setting extra command line arguments)

Definition at line 1178 of file pipeline.py.

Detailed Description

Public Member Functions

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ __repr__()

◆ job()

◆ set_pre_script()

◆ add_pre_script_arg()

◆ set_post_script()

◆ get_post_script()

◆ add_post_script_arg()

◆ get_post_script_arg()

◆ set_name()

◆ get_name()

◆ set_category()

◆ get_category()

◆ set_priority()

◆ get_priority()

◆ add_input_file()

◆ add_output_file()

◆ add_checkpoint_file()

◆ get_input_files()

◆ get_output_files()

◆ get_checkpoint_files()

◆ set_vds_group()

◆ get_vds_group()

◆ add_macro()

◆ add_io_macro()

◆ add_input_macro()

◆ add_output_macro()

◆ add_checkpoint_macro()

◆ get_opts()

◆ add_var_condor_cmd()

◆ add_var_opt()

◆ add_file_opt()

◆ add_var_arg()

◆ add_file_arg()

◆ get_args()

◆ set_retry()

◆ get_retry()

◆ write_job()

◆ write_category()

◆ write_priority()

◆ write_vars()

◆ write_parents()

◆ write_pre_script()

◆ write_post_script()

◆ write_input_files()

◆ write_output_files()

◆ set_log_file()

◆ add_parent()

◆ get_cmd_tuple_list()

◆ get_cmd_line()

◆ finalize()

◆ init()

◆ repr()