Interface definition for online detection pipelines¶

The BAYESTAR rapid localization algorithm is designed as a post-processing stage for compact binary merger search pipelines. The script bayestar-localize-lvalert provides rapid sky localization as a service.

Note

Other available modes of operation for BAYESTAR that are documented elsewhere include the script bayestar-localize-coincs for offline batch processing and the method ligo.skymap.bayestar.localize() for directly calling BAYESTAR from Python code.

Sequence diagram¶

In online operation, search pipelines upload candidates to the Gravitational-Wave Candidate Event Database (GraceDB). The script bayestar-localize-lvalert (or the equivalent Celery task in GWCelery) listens for and intercepts LVAlert pubsub messages. For each event created, the script downloads the pipeline’s output data products from GraceDB, performs rapid sky localization, and uploads the resulting FITS file back to GraceDB.

The interactions between the search pipeline, GraceDB, and BAYESTAR are illustrated in the sequence diagram below. Line styles have the following meanings:

• Solid lines directed into GraceDB represent HTTP requests.

• Solid lines directed out of GraceDB represent HTTP responses.

• Dashed lines represent LVAlert pubsub messages.

sequenceDiagram note over Search: New detection Search ->>+ GraceDB: Upload coinc.xml activate Search note over GraceDB: Create event G123 GraceDB ->>- Search: GraceDB ID: G123 Search ->>+ GraceDB: Upload psd.xml.gz to G123 deactivate Search GraceDB -->>- BAYESTAR: LVAlert: psd.xml.gz added to G123 activate BAYESTAR BAYESTAR ->>+ GraceDB: Get coinc.xml from G123 GraceDB ->>- BAYESTAR: coinc.xml BAYESTAR ->>+ GraceDB: Get psd.xml.gz from G123 GraceDB ->>- BAYESTAR: psd.xml.gz note over BAYESTAR: Perform sky localization BAYESTAR ->> GraceDB: Upload bayestar.fits to G123 deactivate BAYESTAR

Input files¶

This section describes the interface between search pipelines and BAYESTAR. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

The following two files MUST be uploaded to GraceDB:

• coinc.xml: The event file, which SHOULD be the initial upload that creates the event.

• psd.xml.gz: The power spectral density data file, which MUST be uploaded with the psd tag.

The format of both files MUST be LIGO-LW (see LIGO-T990023). LIGO-LW is a legacy XML-based format used by a variety of LIGO/Virgo/KAGRA software and services for storing tabular datasets.

Unfortunately, LIGO-LW is a rather complicated format. We recommend using either the ligo.lw module or GWPy’s tabular LIGO-LW I/O feature to simplify reading and writing LIGO-LW files.

Note

There are two variants of the LIGO-LW format, an old format implemented by glue.ligolw that uses string (“ilwdchar”) row IDs, and a new format implemented by ligo.lw that uses integer row IDs. GraceDB and BAYESTAR can accept either format.

The ligolw_no_ilwdchar command-line tool provided by ligo.lw can convert from the new format to the old format.

The coinc.xml file¶

This file describes the search pipeline’s matched filter output. It MUST contain the point estimates of the time, phase, and amplitude on arrival in each detector. It MUST provide the intrinsic template parameters (masses and spins). It SHOULD contain a signal-to-noise time series for each detector.

The coinc.xml file MUST contain at least the following LIGO-LW tables (in any order):

process
• The process table MUST contain at least one row with the process_id and program columns populated in order to identify the search pipeline.

• The value of those rows’ program column MUST be one of pycbc, gstlal_inspiral, gstlal_inspiral_postcohspiir_online, MBTAOnline, bayestar_realize_coincs, or bayestar-realize-coincs.

• Additional valid columns of this table MAY be populated in order to identify the pipeline software version or include other metadata. Additional unrelated rows (e.g. to identify prior analysis steps such as template bank generation) MAY be included and will be ignored.

sngl_inspiral
coinc
coinc_event_map
coinc_inspiral

The coinc.xml file SHOULD also provide SNR time series for each detector.

• Each SNR time series MUST be stored inside a LIGO_LW element as a serialized COMPLEX8TimeSeries. The function lal.sereries.build_COMPLEX8TimeSeries() can be used to serialize a COMPLEX8TimeSeries.

• Each of the LIGO_LW elements for serialized SNR time series MUST contain a Param element to link it to a row in the sngl_inspiral. The param name MUST be event_id:param and the param value must match the event_id column in the corresponding sngl_inspiral row.

• The SNR time series MUST have an odd number of samples, e.g., the length must be $$2 * n + 1$$ for some integer $$n$$.

• The timestamp of the central sample (e.g. $$n$$ times the sample interval plus the epoch) MUST differ from the corresponding sngl_inspiral row’s time (if present) by no more than one sample interval.

• The timestamps of the samples of the SNR time series MUST correspond to sample boundaries. The timestamps MUST NOT have any sub-sample time shift applied to them.

• For any detector that lacks an SNR time series, sub-sample interpolation SHOULD be applied by the search pipeline to obtain the values for the snr, coa_phase, end_time, and end_time_ns columns in the corresponding row of the sngl_inspiral table.

The psd.xml.gz file¶

This file contains each analyzed detectors’ estimated noise power spectral density (PSD) series.

• There MUST be exactly one PSD per detector analyzed.

• Each PSD MUST be stored inside a LIGO_LW element as a serialized REAL8FrequencySeries. The lal.sereries.build_COMPLEX8TimeSeries() function or the lal.sereries.make_psd_xmldoc() function can be used to serialize REAL8FrequencySeries.

• Each LIGO_LW element MUST contain a Param element to link it to a detector. The param’s name MUST be instrument:param, its type MUST be instrument:param, and its value should be a detector prefix such (e.g. one of H1, L1, V1, K1, I1, etc.)

• Any samples that are invalid because their frequencies are outside of the range analyzed by the search MUST be absent or have their values set to positive infinity. Invalid values MUST NOT be set to zero.

Example files¶

For a minimal example, see the mock coinc.xml and psd.xml.gz files.