Interface definition for online detection pipelines¶
The BAYESTAR rapid localization algorithm is designed as a post-processing stage for compact binary merger search pipelines. The script bayestar-localize-lvalert provides rapid sky localization as a service.
Note
Other available modes of operation for BAYESTAR that are documented
elsewhere include the script bayestar-localize-coincs for offline batch processing
and the method ligo.skymap.bayestar.localize()
for directly calling
BAYESTAR from Python code.
Sequence diagram¶
In online operation, search pipelines upload candidates to the Gravitational-Wave Candidate Event Database (GraceDB). The script bayestar-localize-lvalert (or the equivalent Celery task in GWCelery) listens for and intercepts IGWN Alert messages. For each event created, the script downloads the pipeline’s output data products from GraceDB, performs rapid sky localization, and uploads the resulting FITS file back to GraceDB.
The interactions between the search pipeline, GraceDB, and BAYESTAR are illustrated in the sequence diagram below. Line styles have the following meanings:
Solid lines directed into GraceDB represent HTTP requests.
Solid lines directed out of GraceDB represent HTTP responses.
Dashed lines represent IGWN Alert messages.
Sequence diagram for unified coinc.xml
file¶
Sequence diagram for separate coinc.xml
and psd.xml.gz
files¶
Input files¶
This section describes the interface between search pipelines and BAYESTAR. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
The following file MUST be uploaded to GraceDB:
coinc.xml
: The event file, which SHOULD be the initial upload that creates the event.
The contents of the coinc.xml
file MUST conform to the
Event data section below. The coinc.xml
file SHOULD also contain
the data described in the PSD data section below. If the
coinc.xml
file does not include the PSD data, then the following
additional file MUST be uploaded to GraceDB:
psd.xml.gz
: The power spectral density data file, which MUST be uploaded with thepsd
tag.
If the psd.xml.gz
is uploaded, then its contents MUST conform to the
PSD data section below.
The format of both files MUST be LIGO-LW (see LIGO-T990023). LIGO-LW is a legacy XML-based format used by a variety of LIGO/Virgo/KAGRA software and services for storing tabular datasets.
Unfortunately, LIGO-LW is a rather complicated format. We recommend using
either the ligo.lw
module or GWPy’s tabular LIGO-LW I/O feature to simplify reading and writing LIGO-LW files.
Note
There are two variants of the LIGO-LW format, an old format implemented by
glue.ligolw
that uses string (“ilwdchar”) row IDs, and a new format
implemented by ligo.lw
that uses integer row IDs. GraceDB and
BAYESTAR can accept either format, but pipelines SHOULD upload files in
the new format.
The ligolw_no_ilwdchar command-line tool provided by
ligo.lw
can convert from the new format to the old format.
Event data¶
This event data describes the search pipeline’s matched filter output. It MUST include the point estimates of the time, phase, and amplitude on arrival in each detector. It MUST provide the intrinsic template parameters (masses and spins). It SHOULD include a signal-to-noise time series for each detector.
The event data MUST include at least the following LIGO-LW tables (in any order):
process
The
process
table MUST contain at least one row with theprocess_id
andprogram
columns populated in order to identify the search pipeline.The value of those rows’
program
column MUST be one ofpycbc
,gstlal_inspiral
,gstlal_inspiral_postcohspiir_online
,MBTAOnline
,bayestar_realize_coincs
, orbayestar-realize-coincs
.Additional valid columns of this table MAY be populated in order to identify the pipeline software version or include other metadata. Additional unrelated rows (e.g. to identify prior analysis steps such as template bank generation) MAY be included and will be ignored.
sngl_inspiral
The
sngl_inspiral
table MUST contain exactly one row per detector that the search analyzed.The values of the
event_id
column MUST be distinct across all rows.The values of the following columns that specify the intrinsic template parameters MUST be identical across all rows:
mass1
,mass2
,f_final
,spin1x
,spin1y
,spin1z
,spin2x
,spin2y
, andspin2z
.If the template has zero spin, then the spin columns MAY be left blank. If the template has aligned spins, then the _x_ and _y_ spin components MAY be left blank.
The
end_time
andend_time_ns
columns MUST report the seconds and nanoseconds parts of the GPS time at which the same fiducial reference part of the signal (e.g., the time of merger, or the time at which the inspiral reaches reference frequency) is received in each detector. It SHOULD record the merger time. If the event is an “early warning” or pre-merger event, then it SHOULD record the predicted time of merger.If the event is an early warning event, then the high-frequency cutoff frequency MUST be recorded in the
f_final
column.The
snr
column MUST report the absolute value of the complex matched filter SNR of the best-matching template. It MUST NOT report a modified SNR-like quantity such as newSNR.The
coa_phase
column MUST report the argument of the complex matched filter SNR of the best-matching template.If the search pipeline as identified by the
program
column in theprocess
table ispycbc
, then phase convention of thecoa_phase
column MUST be that the matched filter output is linear in terms of the data. Otherwise, the phase convention MUST be that the matched filter output is antilinear in terms of the data.The
end_time
,end_time_ns
,snr
, andcoa_phase
columns MAY be blank for any row for which there is a corresponding SNR time series (see below).Due to a bug in GraceDB, all columns of the
sngl_inspiral
table (including blank ones) must be present.
coinc
There MUST be exactly one row in the
coinc
table with at least thecoinc_event_id
column populated.The value of the
process_id
column of thecoinc
tale MUST match the value of theprocess_id
column in the:class:process
table that identifies the search pipeline.Note that due to another bug in GraceDB, the
time_slide_id
column MUST be populated. It MAY have a legal dummy value such astime_slide:time_slide_id:0
.
coinc_event_map
There MUST be exactly one row in the
coinc_event_map
table for each row in thesngl_inspiral
table.The value in each row’s
event_id
column must be set to the value of theevent_id
column in the corresponding row of thesngl_inspiral
table.The value in each row’s
table_name
column must be setsngl_inspiral
.Each row MUST have the
coinc_event_id
column set to the value of thecoinc_event_id
column in the one row of thecoinc
table.
coinc_inspiral
The
coinc_inspiral
table MUST be present because it is required by GraceDB (although it is ignored by BAYESTAR).It MUST have exactly one row.
The value in the
coinc_event_id
column MUST match the value in the corresponding column in thecoinc
table.The following columns MUST be populated:
coinc_event_id
,combined_far
,end_time
,end_time_ns
,ifos
, andsnr
.The
mass
andmchirp
columns SHOULD be populated.
The coinc.xml
file SHOULD also provide SNR time series for each
detector.
Each SNR time series MUST be stored inside a
LIGO_LW
element as a serializedCOMPLEX8TimeSeries
. The functionlal.sereries.build_COMPLEX8TimeSeries()
can be used to serialize aCOMPLEX8TimeSeries
.Each of the
LIGO_LW
elements for serialized SNR time series MUST contain aParam
element to link it to a row in thesngl_inspiral
. The param name MUST beevent_id:param
and the param’s type and value must match theevent_id
column in the correspondingsngl_inspiral
row.The SNR time series MUST have an odd number of samples, e.g., the length must be \(2 * n + 1\) for some integer \(n\).
The timestamp of the central sample (e.g. \(n\) times the sample interval plus the epoch) MUST differ from the corresponding
sngl_inspiral
row’s time (if present) by no more than one sample interval.The timestamps of the samples of the SNR time series MUST correspond to sample boundaries. The timestamps MUST NOT have any sub-sample time shift applied to them.
For any detector that lacks an SNR time series, sub-sample interpolation SHOULD be applied by the search pipeline to obtain the values for the
snr
,coa_phase
,end_time
, andend_time_ns
columns in the corresponding row of thesngl_inspiral
table.
PSD data¶
The PSD data consists of each analyzed detectors’ estimated noise power spectral density (PSD) series.
There MUST be exactly one PSD per detector analyzed.
Each PSD MUST be stored inside a
LIGO_LW
element as a serializedREAL8FrequencySeries
. Thelal.sereries.build_COMPLEX8TimeSeries()
function or thelal.sereries.make_psd_xmldoc()
function can be used to serializeREAL8FrequencySeries
.Each
LIGO_LW
element MUST contain aParam
element to link it to a detector. The param’s name MUST beinstrument:param
, its type MUST beinstrument:param
, and its value should be a detector prefix such (e.g. one ofH1
,L1
,V1
,K1
,I1
, etc.)Any samples that are invalid because their frequencies are outside of the range analyzed by the search MUST be absent or have their values set to positive infinity. Invalid values MUST NOT be set to zero.
Example files¶
For a minimal example, see the mock coinc.xml
file.