lal.pipeline.CondorDAG Class Reference

Detailed Description

A CondorDAG is a Condor Directed Acyclic Graph that describes a collection of Condor jobs and the order in which to run them.

All Condor jobs in the DAG must write their Codor logs to the same file. NOTE: The log file must not be on an NFS mounted system as the Condor jobs must be able to get an exclusive file lock on the log file.

Definition at line 1245 of file pipeline.py.

Inherits object.

Public Member Functions
def	__init__ (self, log)
def	get_nodes (self)
	Return a list containing all the nodes in the DAG. More...
def	get_jobs (self)
	Return a list containing all the jobs in the DAG. More...
def	set_integer_node_names (self)
	Use integer node names for the DAG. More...
def	set_dag_file (self, path)
	Set the name of the file into which the DAG is written. More...
def	get_dag_file (self)
	Return the path to the DAG file. More...
def	add_node (self, node)
	Add a CondorDAGNode to this DAG. More...
def	add_maxjobs_category (self, categoryName, maxJobsNum)
	Add a category to this DAG called categoryName with a maxjobs of maxJobsNum. More...
def	get_maxjobs_categories (self)
	Return an array of tuples containing (categoryName,maxJobsNum) More...
def	write_maxjobs (self, fh, category)
	Write the DAG entry for this category's maxjobs to the DAG file descriptor. More...
def	write_sub_files (self)
	Write all the submit files used by the dag to disk. More...
def	write_concrete_dag (self)
	Write all the nodes in the DAG to the DAG file. More...
def	write_dag (self)
	Write a dag. More...
def	write_script (self)
	Write the workflow to a script (.sh instead of .dag). More...

Constructor & Destructor Documentation

__init__()

def lal.pipeline.CondorDAG.__init__	(		self,
			log
	)

Parameters

log	path to log file which must not be on an NFS mounted file system.

Definition at line 1249 of file pipeline.py.

Member Function Documentation

get_nodes()

def lal.pipeline.CondorDAG.get_nodes

(

self

)

Return a list containing all the nodes in the DAG.

Definition at line 1262 of file pipeline.py.

get_jobs()

def lal.pipeline.CondorDAG.get_jobs

(

self

)

Return a list containing all the jobs in the DAG.

Definition at line 1268 of file pipeline.py.

set_integer_node_names()

def lal.pipeline.CondorDAG.set_integer_node_names

(

self

)

Use integer node names for the DAG.

Definition at line 1274 of file pipeline.py.

set_dag_file()

def lal.pipeline.CondorDAG.set_dag_file	(		self,
			path
	)

Set the name of the file into which the DAG is written.

Parameters

path	path to DAG file.

Definition at line 1281 of file pipeline.py.

get_dag_file()

def lal.pipeline.CondorDAG.get_dag_file

(

self

)

Return the path to the DAG file.

Definition at line 1287 of file pipeline.py.

add_node()

def lal.pipeline.CondorDAG.add_node	(		self,
			node
	)

Add a CondorDAGNode to this DAG.

The CondorJob that the node uses is also added to the list of Condor jobs in the DAG so that a list of the submit files needed by the DAG can be maintained. Each unique CondorJob will be added once to prevent duplicate submit files being written.

Parameters

node	CondorDAGNode to add to the CondorDAG.

Definition at line 1300 of file pipeline.py.

add_maxjobs_category()

def lal.pipeline.CondorDAG.add_maxjobs_category	(		self,
			categoryName,
			maxJobsNum
	)

Add a category to this DAG called categoryName with a maxjobs of maxJobsNum.

Parameters

categoryName	category name
maxJobsNum	max jobs num

Definition at line 1317 of file pipeline.py.

get_maxjobs_categories()

def lal.pipeline.CondorDAG.get_maxjobs_categories

(

self

)

Return an array of tuples containing (categoryName,maxJobsNum)

Definition at line 1323 of file pipeline.py.

write_maxjobs()

def lal.pipeline.CondorDAG.write_maxjobs	(		self,
			fh,
			category
	)

Write the DAG entry for this category's maxjobs to the DAG file descriptor.

Parameters

fh	descriptor of open DAG file.
category	tuple containing type of jobs to set a maxjobs limit for and the maximum number of jobs of that type to run at once.

Definition at line 1332 of file pipeline.py.

write_sub_files()

def lal.pipeline.CondorDAG.write_sub_files

(

self

)

Write all the submit files used by the dag to disk.

Each submit file is written to the file name set in the CondorJob.

Definition at line 1339 of file pipeline.py.

write_concrete_dag()

def lal.pipeline.CondorDAG.write_concrete_dag

(

self

)

Write all the nodes in the DAG to the DAG file.

Definition at line 1349 of file pipeline.py.

write_dag()

def lal.pipeline.CondorDAG.write_dag

(

self

)

Write a dag.

Definition at line 1376 of file pipeline.py.

write_script()

def lal.pipeline.CondorDAG.write_script

(

self

)

Write the workflow to a script (.sh instead of .dag).

Assuming that parents were added to the DAG before their children, dependencies should be handled correctly.

Definition at line 1388 of file pipeline.py.