Data Generation

Module to generate/prepare data, likelihood, and priors for parallel runs.

This will create a directory structure for your parallel runs to store the output files, logs and plots. It will also generate a data_dump that stores information on the run settings and data to be analysed.

Note that several of these arguments are inherited from bilby_pipe.

Command line interface for data generation

usage: parallel_bilby_generation [--version] [-n NLIVE] [--dlogz DLOGZ]
                                 [--n-effective N_EFFECTIVE]
                                 [--dynesty-sample DYNESTY_SAMPLE]
                                 [--dynesty-bound DYNESTY_BOUND]
                                 [--walks WALKS] [--maxmcmc MAXMCMC]
                                 [--nact NACT] [--min-eff MIN_EFF]
                                 [--facc FACC] [--vol-dec VOL_DEC]
                                 [--vol-check VOL_CHECK] [--enlarge ENLARGE]
                                 [--n-check-point N_CHECK_POINT]
                                 [--max-its MAX_ITS]
                                 [--max-run-time MAX_RUN_TIME]
                                 [--fast-mpi FAST_MPI]
                                 [--mpi-timing MPI_TIMING]
                                 [--mpi-timing-interval MPI_TIMING_INTERVAL]
                                 [--nestcheck] [--nsamples NSAMPLES]
                                 [--ntemps NTEMPS] [--nwalkers NWALKERS]
                                 [--max-iterations MAX_ITERATIONS]
                                 [--ncheck NCHECK]
                                 [--burn-in-nact BURN_IN_NACT]
                                 [--thin-by-nact THIN_BY_NACT]
                                 [--frac-threshold FRAC_THRESHOLD]
                                 [--nfrac NFRAC] [--min-tau MIN_TAU]
                                 [--Tmax TMAX] [--safety SAFETY]
                                 [--autocorr-c AUTOCORR_C]
                                 [--autocorr-tol AUTOCORR_TOL] [--adapt]
                                 [--sampling-seed SAMPLING_SEED] [-c]
                                 [--no-plot] [--do-not-save-bounds-in-resume]
                                 [--check-point-deltaT CHECK_POINT_DELTAT]
                                 [--rotate-checkpoints] [-h] [-v]
                                 [--calibration-model {CubicSpline,None}]
                                 [--spline-calibration-envelope-dict SPLINE_CALIBRATION_ENVELOPE_DICT]
                                 [--spline-calibration-nodes SPLINE_CALIBRATION_NODES]
                                 [--spline-calibration-amplitude-uncertainty-dict SPLINE_CALIBRATION_AMPLITUDE_UNCERTAINTY_DICT]
                                 [--spline-calibration-phase-uncertainty-dict SPLINE_CALIBRATION_PHASE_UNCERTAINTY_DICT]
                                 [--ignore-gwpy-data-quality-check IGNORE_GWPY_DATA_QUALITY_CHECK]
                                 [--gps-tuple GPS_TUPLE] [--gps-file GPS_FILE]
                                 [--timeslide-file TIMESLIDE_FILE]
                                 [--timeslide-dict TIMESLIDE_DICT]
                                 [--trigger-time TRIGGER_TIME]
                                 [--n-simulation N_SIMULATION]
                                 [--data-dict DATA_DICT]
                                 [--data-format DATA_FORMAT]
                                 [--allow-tape ALLOW_TAPE]
                                 [--channel-dict CHANNEL_DICT]
                                 [--gaussian-noise | --zero-noise]
                                 [--coherence-test] [--detectors DETECTORS]
                                 [--duration DURATION]
                                 [--generation-seed GENERATION_SEED]
                                 [--psd-dict PSD_DICT]
                                 [--psd-fractional-overlap PSD_FRACTIONAL_OVERLAP]
                                 [--post-trigger-duration POST_TRIGGER_DURATION]
                                 [--sampling-frequency SAMPLING_FREQUENCY]
                                 [--psd-length PSD_LENGTH]
                                 [--psd-maximum-duration PSD_MAXIMUM_DURATION]
                                 [--psd-method PSD_METHOD]
                                 [--psd-start-time PSD_START_TIME]
                                 [--maximum-frequency MAXIMUM_FREQUENCY]
                                 [--minimum-frequency MINIMUM_FREQUENCY]
                                 [--tukey-roll-off TUKEY_ROLL_OFF]
                                 [--resampling-method {lal,gwpy}]
                                 [--injection-dict INJECTION_DICT | --injection-file INJECTION_FILE]
                                 [--injection-numbers INJECTION_NUMBERS]
                                 [--injection-waveform-approximant INJECTION_WAVEFORM_APPROXIMANT]
                                 [--label LABEL] [--outdir OUTDIR]
                                 [--periodic-restart-time PERIODIC_RESTART_TIME]
                                 [--scheduler-analysis-time SCHEDULER_ANALYSIS_TIME]
                                 [--condor-job-priority CONDOR_JOB_PRIORITY]
                                 [--log-directory LOG_DIRECTORY]
                                 [--analysis-executable ANALYSIS_EXECUTABLE]
                                 [--distance-marginalization-lookup-table DISTANCE_MARGINALIZATION_LOOKUP_TABLE]
                                 [--jitter-time JITTER_TIME]
                                 [--reference-frame REFERENCE_FRAME]
                                 [--time-reference TIME_REFERENCE]
                                 [--likelihood-type LIKELIHOOD_TYPE]
                                 [--roq-folder ROQ_FOLDER]
                                 [--roq-weights ROQ_WEIGHTS]
                                 [--roq-scale-factor ROQ_SCALE_FACTOR]
                                 [--extra-likelihood-kwargs EXTRA_LIKELIHOOD_KWARGS]
                                 [--create-plots] [--create-summary]
                                 [--notification NOTIFICATION]
                                 [--existing-dir EXISTING_DIR]
                                 [--webdir WEBDIR]
                                 [--summarypages-arguments SUMMARYPAGES_ARGUMENTS]
                                 [--result-format {json,hdf5,pickle}]
                                 [--default-prior DEFAULT_PRIOR]
                                 [--deltaT DELTAT]
                                 [--prior-file PRIOR_FILE | --prior-dict PRIOR_DICT]
                                 [--convert-to-flat-in-component-mass CONVERT_TO_FLAT_IN_COMPONENT_MASS]
                                 [--single-postprocessing-executable SINGLE_POSTPROCESSING_EXECUTABLE]
                                 [--single-postprocessing-arguments SINGLE_POSTPROCESSING_ARGUMENTS]
                                 [--n-parallel N_PARALLEL]
                                 [--waveform-generator WAVEFORM_GENERATOR]
                                 [--reference-frequency REFERENCE_FREQUENCY]
                                 [--waveform-approximant WAVEFORM_APPROXIMANT]
                                 [--catch-waveform-errors CATCH_WAVEFORM_ERRORS]
                                 [--pn-spin-order PN_SPIN_ORDER]
                                 [--pn-tidal-order PN_TIDAL_ORDER]
                                 [--pn-phase-order PN_PHASE_ORDER]
                                 [--pn-amplitude-order PN_AMPLITUDE_ORDER]
                                 [--numerical-relativity-file NUMERICAL_RELATIVITY_FILE]
                                 [--waveform-arguments-dict WAVEFORM_ARGUMENTS_DICT]
                                 [--mode-array MODE_ARRAY]
                                 [--frequency-domain-source-model FREQUENCY_DOMAIN_SOURCE_MODEL]
                                 [--conversion-function CONVERSION_FUNCTION]
                                 [--generation-function GENERATION_FUNCTION]
                                 [--sampler {dynesty,ptemcee}] --nodes NODES
                                 --ntasks-per-node NTASKS_PER_NODE --time TIME
                                 [--mem-per-cpu MEM_PER_CPU]
                                 [--extra-lines EXTRA_LINES]
                                 [--slurm-extra-lines SLURM_EXTRA_LINES]

Positional Arguments


Configuration ini file

Named Arguments


show program’s version number and exit

-v, --verbose

Verbose output

Default: False


If true, use simulated Gaussian noise

Default: False


Use a zero noise realisation

Default: False


A single injection dictionary given in the ini file


Injection file to use. See bilby_pipe_create_injection_file –help for supported formats


The prior file


A dictionary of priors (alternative to prior-file). Multiline dictionaries are supported, but each line must contain a singleparameter specification and finish with a comma.


Possible choices: dynesty, ptemcee

The parallelised sampler to use, defaults to dynesty

Default: “dynesty”

Dynesty Settings

-n, --nlive

Number of live points

Default: 1000


Stopping criteria: remaining evidence, (default=0.1)

Default: 0.1


Stopping criteria: effective number of samples, (default=inf)

Default: inf


Dynesty sampling method (default=rwalk). Note, the dynesty rwalk method is overwritten by parallel bilby for an optimised version

Default: “rwalk”


Dynesty bounding method (default=multi)

Default: “multi”


Minimum number of walks, defaults to 100

Default: 100


Maximum number of walks, defaults to 5000

Default: 5000


Number of autocorrelation times to take, defaults to 5

Default: 5


The minimum efficiency at which to switch from uniform sampling.

Default: 10


See dynesty.NestedSampler

Default: 0.5


See dynesty.NestedSampler

Default: 0.5


See dynesty.NestedSampler

Default: 8


See dynesty.NestedSampler

Default: 1.5


Steps to take before attempting checkpoint

Default: 100


Maximum number of iterations to sample for (default=1.e10)

Default: 10000000000


Maximum time to run for (default=1.e10 s)

Default: 10000000000.0


Fast MPI communication pattern (default=False)

Default: False


Print MPI timing when finished (default=False)

Default: False


Interval to write timing snapshot to disk (default=0 – disabled)

Default: 0


Save a ‘nestcheck’ pickle in the outdir (default=False). This pickle stores a nestcheck.data_processing.process_dynesty_run object, which can be used during post processing to compute the implementation and bootstrap errors explained by Higson et al (2018) in “Sampling Errors In Nested Sampling Parameter Estimation”.

Default: False

PTEmcee Settings


Number of samples to draw

Default: 10000


Number of temperatures

Default: 20


Number of walkers

Default: 100


Maximum number of iterations

Default: 100000


Period with which to check convergence

Default: 500


Number of autocorrelation times to discard for burn-in

Default: 50.0


Thin-by number of autocorrelation times

Default: 1.0


Threshold on the fractional change in ACT required for convergence

Default: 0.01


The number of checks passing the frac-threshold for convergence

Default: 5


The minimum tau to accept: used to prevent early convergence

Default: 30


The maximum temperature to use, default=10000

Default: 10000


Multiplicitive safety factor on the estimated tau

Default: 1.0


The step size for the window search when calculating tau. Default: 5

Default: 5.0


The minimum number of autocorrelations needs to trust the autocorrelation estimate. Default: 0 (always return a result)

Default: 50.0


If True, the temperature ladder is dynamically adapted as the sampler runs to achieve uniform swap acceptance ratios between adjacent chains. See arXiv:1501.05823 for details.

Default: False

Misc. Settings


Default: False


Random seed for sampling, parallel runs will be incremented

-c, --clean

Run clean: ignore any resume files

Default: False


If true, don’t generate check-point plots

Default: False


If true, do not store bounds in the resume file. This can make resume files large (~GB)

Default: False


Write a checkpoint resume file and diagnostic plots every deltaT [s].

Default: 3600


If true, backup checkpoint before overwriting (ending in ‘.bk’).

Default: False

Calibration arguments

Which calibration model and settings to use.


Possible choices: CubicSpline, None

Choice of calibration model, if None, no calibration is used


Dictionary pointing to the spline calibration envelope files


Number of calibration nodes

Default: 5


Dictionary of the amplitude uncertainties for the constant uncertainty model


Dictionary of the phase uncertainties for the constant uncertainty model

Data generation arguments

How to generate the data, e.g., from a list of gps times or simulated Gaussian noise.


Ignores the check to see if data queried from GWpy (ie not gaussian noise) is obtained from time when the IFOs are in science mode.

Default: True


Tuple of the (start, step, number) of GPS start times. For example, (10, 1, 3) produces the gps start times [10, 11, 12]. If given, gps-file is ignored.


File containing segment GPS start times. This can be a multi-column file if (a) it is comma-separated and (b) the zeroth column contains the gps-times to use


File containing detector timeslides. Requires a GPS time file to also be provided. One column for each detector. Order of detectors specified by –detectors argument. Number of timeslides must correspond to the number of GPS times provided.


Dictionary containing detector timeslides: applies a fixed offset per detector. E.g. to apply +1s in H1, {H1: 1}


Either a GPS trigger time, or the event name (e.g. GW150914). For event names, the gwosc package is used to identify the trigger time


Number of simulated segments to use with gaussian-noise Note, this must match the number of injections specified

Default: 0


Dictionary of paths to gwf, or hdf5 data files


If given, the data format to pass to `, see


If true (default), allow reading data from tape. See `gwpy.timeseries.TimeSeries.get() for more information.

Default: True


Channel dictionary: keys relate to the detector with values the channel name, e.g. ‘GDS-CALIB_STRAIN’. For GWOSC open data, set the channel-dict keys to ‘GWOSC’. Note, the dictionary should follow basic python dict syntax.

Detector arguments

How to set up the interferometers and power spectral density.


Run the analysis for all detectors together and for each detector separately

Default: False


The names of detectors to use. If given in the ini file, detectors are specified by detectors=[H1, L1]. If given at the command line, as –detectors H1 –detectors L1


The duration of data around the event to use

Default: 4


Random seed used during data generation. If no generation seed provided, a random seed between 1 and 1e6 is selected. If a seed is provided, it is used as the base seed and all generation jobs will have their seeds set as {generation_seed = base_seed + job_idx}.


Dictionary of PSD files to use


Fractional overlap of segments used in estimating the PSD

Default: 0.5


Time (in s) after the trigger_time to the end of the segment

Default: 2.0


Default: 4096


Sets the psd duration (up to the psd-duration-maximum). PSD duration calculated by psd-length x duration [s]. Default is 32.

Default: 32


The maximum allowed PSD duration in seconds, default is 1024s.

Default: 1024


PSD method see gwpy.timeseries.TimeSeries.psd for options

Default: “median”


Start time of data (relative to the segment start) used to generate the PSD. Defaults to psd-duration before the segment start time


The maximum frequency, given either as a float for all detectors or as a dictionary (see minimum-frequency)


The minimum frequency, given either as a float for all detectors or as a dictionary where all keys relate the detector with values of the minimum frequency, e.g. {H1: 10, L1: 20}. If the waveform generation should start the minimum frequency for any of the detectors, add another entry to the dictionary, e.g., {H1: 40, L1: 60, waveform: 20}.

Default: “20”


Roll off duration of tukey window in seconds, default is 0.4s

Default: 0.4


Possible choices: lal, gwpy

Resampling method to use: lal matches the resampling used by lalinference/BayesWave

Default: “lal”

Injection arguments

Whether to include software injections and how to generate them.


Create data from an injection file

Default: False


Specific injections rows to use from the injection_file, e.g. `injection_numbers=[0,3] selects the zeroth and third row


The name of the waveform approximant to use to create injections. If none is specified, then the waveform-approximant will be usedas the injection-waveform-approximant.

Job submission arguments

How the jobs should be formatted, e.g., which job scheduler to use.


Output label

Default: “label”


The output directory. If outdir already exists, an auto-incrementing naming scheme is used

Default: “outdir”


If given, overwrite the outdir (if it exists)

Default: False


Time after which the job will self-evict when scheduler=condor. After this, condor will restart the job. Default is 28800. This is used to decrease the chance of HTCondor hard evictions

Default: 28800


Default: 7-00:00:00


Attempt to submit the job after the build

Default: False


Job priorities allow a user to sort their HTCondor jobs to determine which are tried to be run first. A job priority can be any integer: larger values denote better priority. By default HTCondor job priority=0.

Default: 0


If given, an alternative path for the log output


Path to an executable to replace bilby_pipe_analysis, be aware that this executable will pass the complete ini file (in the outdir.)

Likelihood arguments

Options for setting up the likelihood.


Boolean. If true, use a distance-marginalized likelihood

Default: False


Path to the distance-marginalization lookup table


Boolean. If true, use a phase-marginalized likelihood

Default: False


Boolean. If true, use a time-marginalized likelihood

Default: False


Boolean. If true, and using a time-marginalized likelihood ‘time jittering’ will be performed

Default: True


Reference frame for the sky parameterisation, either ‘sky’ (default) or, e.g., ‘H1L1’

Default: “sky”


Time parameter to sample in, either ‘geocent’ (default) or, e.g., ‘H1’

Default: “geocent”


The likelihood. Can be one of [GravitationalWaveTransient, ROQGravitationalWaveTransient, zero] or python path to a bilby likelihood class available in the users installation. The –roq-folder is required if the ROQ likelihood used.The –roq-folder is required if the ROQ likelihood used.If zero is given, a testing ZeroLikelihood is used which alwaysreturn zero.

Default: “GravitationalWaveTransient”


The data for ROQ


If given, the ROQ weights to use (rather than building them). This must be given along with the roq-folder for checking


Rescaling factor for the ROQ, default is 1 (no rescaling)

Default: 1


Additional keyword arguments to pass to the likelihood. Any arguments which are named bilby_pipe arguments, e.g., distance_marginalization should NOT be included. This is only used if you are not using the GravitationalWaveTransient or ROQGravitationalWaveTransient likelihoods

Output arguments

What kind of output/summary to generate.


Create diagnostic and posterior plots

Default: False


Create a PESummary page

Default: False


Notification setting for HTCondor jobs. One of ‘Always’,’Complete’,’Error’,’Never’. If defined by ‘Always’, the owner will be notified whenever the job produces a checkpoint, as well as when the job completes. If defined by ‘Complete’, the owner will be notified when the job terminates. If defined by ‘Error’, the owner will only be notified if the job terminates abnormally, or if the job is placed on hold because of a failure, and not by user request. If defined by ‘Never’ (the default), the owner will not receive e-mail, regardless to what happens to the job. Note, an email arg is also required for notifications to be emailed.

Default: Never


If given, add results to an directory with an an existing summary.html file


Directory to store summary pages. If not given, defaults to outdir/results_page


Arguments (in the form of a dictionary) to pass to the summarypages executable


Possible choices: json, hdf5, pickle

Format to save the result file in.

Default: “json”

Prior arguments

Specify the prior settings.


The name of the prior set to base the prior on. Can be one of[PriorDict, BBHPriorDict, BNSPriorDict, CalibrationPriorDict]

Default: “BBHPriorDict”


The symmetric width (in s) around the trigger time to search over the coalesence time

Default: 0.2


Convert a flat-in chirp mass and mass-ratio prior file to flat in component mass during the post-processing. Note, the prior must be uniform in Mc and q with constraints in m1 and m2 for this to work

Default: False

Post processing arguments

What post-processing to perform.


An executable name for postprocessing. A single postprocessing job is run as a child for each analysis jobs: note the difference with respect postprocessing-executable


Arguments to pass to the single postprocessing executable. The str ‘$RESULT’ will be replaced by the path to the individual result file

Sampler arguments


Number of identical parallel jobs to run per event

Default: 1

Waveform arguments

Setting for the waveform generator


The waveform generator class, should be a python path. This will not be able to use any arguments not passed to the default.

Default: “”


The reference frequency

Default: 20


The name of the waveform approximant to use for PE.

Default: “IMRPhenomPv2”


Turns on waveform error catching

Default: False


Post-newtonian order to use for the spin

Default: -1


Post-Newtonian order to use for tides

Default: -1


post-Newtonian order to use for the phase

Default: -1


Post-Newtonian order to use for the amplitude. Also used to determine the waveform starting frequency.

Default: 0


Path to a h5 numerical relativity file to inject, see for examples


A dictionary of arbitrary additional waveform-arguments to pass to the bilby waveform generator’s waveform_arguments


Array of modes to use for the waveform. Should be a list of lists, eg. [[2,2], [2,-2]]


Name of the frequency domain source model. Can be one of[lal_binary_black_hole, lal_binary_neutron_star,lal_eccentric_binary_black_hole_no_spins, sinegaussian, supernova, supernova_pca_model] or any python path to a bilby source function the users installation, e.g. examp.source.bbh

Default: “lal_binary_black_hole”


Optional python path to a user-specified conversion function If unspecified, this is determined by the frequency_domain_source_model.If the source-model contains binary_black_hole, the conversion function is If the source-model contains binary_neutron_star, the generation function is If you specify your own function, you may wish to use the I/O of those functions as templates


Optional python path to a user-specified generation function If unspecified, this is determined by the frequency_domain_source_model.If the source-model contains binary_black_hole, the generation function is If the source-model contains binary_neutron_star, the generation function is If you specify your own function, you may wish to use the I/O of those functions as templates

Slurm Settings


Number of nodes to use


Number of tasks per node


Maximum wall time (defaults to 24:00:00)

Default: “24:00:00”


Memory per CPU (defaults to None)


Additional lines, separated by ‘;’, use for setting up conda env


additional slurm args (args that need #SBATCH in front) of the form arg=val separated by sapce