gstlal_fake_frames ================== This program will make fake data in a variety of ways. Its input is anything supported by datasource.py. You can additionally whiten the data or apply a frequency domain coloring filter. It writes its output to frame files. Overview graph of the pipeline ------------------------------ Gray boxes are optional and depend on the command line given .. graphviz:: digraph G { // graph properties rankdir=LR; compound=true; node [shape=record fontsize=10 fontname="Verdana"]; edge [fontsize=8 fontname="Verdana"]; // nodes "mkbasicsrc()" [URL="\ref datasource.mkbasicsrc()"]; "whitened_multirate_src()" [label="whitened_multirate_src()", URL="\ref multirate_datasource.mkwhitened_multirate_src()", style=filled, color=lightgrey]; capsfilter [style=filled, color=lightgrey, URL="\ref pipeparts.mkcapsfilter()"]; resample [style=filled, color=lightgrey, URL="\ref pipeparts.mkresample()"]; audioamplify [style=filled, color=lightgrey, URL="\ref pipeparts.mkaudioamplify()", label="audioamplify \n if --data-source=white \n in order to have unit variance \n after resampling"]; lal_shift[style=filled, color=lightgrey, URL="\ref pipeparts.mkshift()", label="lal_shift \n [iff --shift provided]"]; progressreport1 [URL="\ref pipeparts.mkprogressreport()", label="progressreport \n [iff --verbose provided]"]; progressreport2 [style=filled, color=lightgrey, URL="\ref pipeparts.mkprogressreport()", label="progressreport \n [iff --verbose provided]"]; lal_firbank [style=filled, color=lightgrey, URL="\ref pipeparts.mkfirbank()", label="lal_firbank \n [iff --color-psd provided]"]; taginject [URL="\ref pipeparts.mktaginject()"]; lal_simulation [style=filled, color=lightgrey, URL="\ref pipeparts.mkinjections()", label="lal_simulation \n [iff --injections provided]"]; framecppchannelmux [URL="\ref pipeparts.mkframecppchannelmux()"]; framecppfilesink [URL="\ref pipeparts.mkframecppfilesink()"]; // connect it up "mkbasicsrc()" -> progressreport1-> lal_shift -> progressreport2; progressreport2 -> "whitened_multirate_src()" [label="if --whiten given"]; "whitened_multirate_src()" -> lal_firbank; progressreport2 -> resample [label="if --whiten not given"]; resample -> capsfilter; capsfilter -> audioamplify audioamplify -> lal_firbank; lal_firbank -> taginject -> lal_simulation -> framecppchannelmux -> framecppfilesink; } Usage cases ----------- 1. Making initial LIGO colored noise, Advanced LIGO noise, etc for different instruments. See datasource.append_options() for other choices:: $ gstlal_fake_frames --data-source=LIGO --channel-name=H1=FAKE-STRAIN \ --frame-type=H1_FAKE --gps-start-time=900000000 --gps-end-time=900005000 \ --output-path=testframes --verbose $ gstlal_fake_frames --data-source=AdvLIGO --channel-name=L1=FAKE-STRAIN \ --frame-type=L1_FAKE --gps-start-time=900000000 --gps-end-time=900005000 \ --output-path=testframes --verbose 2. Custom colored noise. Obviously it is not practical to code up every possible noise curve to simulate as a custom data source. However, it is possible to provide your own custom noise curve as an ASCII file with frequency in one column and strain/Hz in the second. e.g., early advanced ligo noise curve https://git.ligo.org/lscsoft/gstlal/raw/master/gstlal/share/early_aligo_asd.txt You will need to convert ASD text files to PSD xml files using gstlal_psd_xml_from_asd_txt first.:: $ gstlal_fake_frames --data-source=white --color-psd v1psd.xml.gz \ --channel-name=V1=FAKE-STRAIN --frame-type=V1_FAKE --gps-start-time=900000000 \ --gps-end-time=900005000 --output-path=testframes --verbose 3. Recoloring existing frame data This procedure is very similar to the above except that instead of using white noise to drive the fake frame generation, we start with real frame data and whiten it. Recoloring can be thought of as first whitening data and then applying a coloring filter. You must first make a reference PSD of the data with e.g. gstlal_reference_psd. You will need to make a frame cache of the frames to recolor.:: $ gstlal_fake_frames --whiten-reference-psd reference_psd.xml.gz \ --color-psd recolor_psd.xml.gz --data-source frames \ --output-path /path/to/output --output-channel-name h_16384Hz \ --gps-start-time 966384031 --frame-type T1300121_V1_EARLY_RECOLORED_V2 \ --gps-end-time 966389031 --frame-duration 16 --frames-per-file 256 \ --frame-cache frame.cache --channel-name=V1=h_16384Hz --verbose 4. Creating injections into silence (assumes you have an injection xml file from e.g. lalapps_inspinj):: $ gstlal_fake_frames --data-source silence --output-path /path/to/output \ --gps-start-time 966384031 --frame-type V1_INJECTIONS \ --gps-end-time 966389031 --frame-duration 16 --frames-per-file 256 \ --verbose --channel-name=V1=INJECTIONS --injections injections.xml 5. Other things are certainly possible, please add some! Debug ----- It is possible to check the pipeline graph by interupting the running process with ctrl+c if you have the GST_DEBUG_DUMP_DOT_DIR evironment set. A dot graph will be written to gstlal_fake_frames. Here is an example:: $ GST_DEBUG_DUMP_DOT_DIR="." gstlal_fake_frames --data-source=LIGO \ --channel-name=H1=FAKE-STRAIN --frame-type=H1_FAKE \ --gps-start-time=900000000 --gps-end-time=900005000 \ --output-path=testframes --verbose`` You should see:: Wrote pipeline to ./gstlal_fake_frames_PLAYING.dot After Ctrl+c you should see:: ^C*** SIG 2 attempting graceful shutdown (this might take several minutes) ... *** Wrote pipeline to ./gstlal_fake_frames.dot You can then turn the pipeline graph into an image with dot, e.g.,:: $ dot -Tpng gstlal_fake_frames_PLAYING.dot > test.png Review ------ Command line options -------------------- .. code-block:: none Usage: gstlal_fake_frames [options] Options: -h, --help show this help message and exit --shift=ns Number of nanoseconds to delay (negative) or advance (positive) the time stream --sample-rate=Hz Sample rate at which to generate the data, should be less than or equal to the sample rate of the measured psds provided, default = 16384 Hz, max 16384 Hz --whiten-reference-psd=name Set the name of psd xml file to whiten the data with --whiten-track-psd Whiten the data by tracking PSD changes, can be combined with --whiten-reference-psd to seed the process. --color-psd=name Set the name of psd xml file to color the data with --output-path=name Path to output frame files (default = "."). --output-channel-name=name The name of the channel in the output frames. The default is the same as the channel name --output-frame-type=name Frame type, required --output-frame-duration=s Set the duration of the output frames. The duration of the frame file will be multiplied by --frames-per- file. Default: 16s --output-frames-per-file=n Set the number of frames per file. Default: 256 -v, --verbose Be verbose (optional). Data source options: Use these options to set up the appropriate data source --data-source=source Set the data source from [frames|framexmit|lvshm|nds|silence|white]. Required. --block-size=bytes Data block size to read in bytes. Default 16384 * 8 * 512 (512 seconds of double precision data at 16384 Hz. This parameter is only used if --data-source is one of white, silence, AdvVirgo, LIGO, AdvLIGO, nds. --frame-cache=filename Set the name of the LAL cache listing the LIGO-Virgo .gwf frame files (optional). --frame-type=name Set the frame type for a given instrument. Can be given multiple times as --frame-type=IFO=FRAME-TYPE. Used with --data-source=frames --data-find-server=url Set the data find server for LIGO data discovery. Used with --data-source=frames --gps-start-time=seconds Set the start time of the segment to analyze in GPS seconds. Required unless --data-source=lvshm --gps-end-time=seconds Set the end time of the segment to analyze in GPS seconds. Required unless --data-source=lvshm --injection-file=filename Set the name of the LIGO light-weight XML file from which to load injections (optional). --channel-name=name Set the name of the channels to process. Can be given multiple times as --channel-name=IFO=CHANNEL-NAME --idq-channel-name=idqname iDQ channel names to process. Must also provide idq- state-channel-name. Can be given multiple times as --idq-channel-name=IFO=IDQ-CHANNEL-NAME --idq-state-channel-name=idqstatename iDQ state channel names to process. Can be given multiple times as --idq-state-channel-name=IFO=IDQ- STATE-CHANNEL-NAME --nds-host=hostname Set the remote host or IP address that serves nds data. This is required iff --data-source=nds --nds-port=portnumber Set the port of the remote host that serves nds data. This is required iff --data-source=nds --nds-channel-type=type Set the port of the remote host that serves nds data. This is required only if --data-source=nds. default==online --framexmit-addr=name Set the address of the framexmit service. Can be given multiple times as --framexmit- addr=IFO=xxx.xxx.xxx.xxx:port --framexmit-iface=name Set the multicast interface address of the framexmit service. --state-channel-name=name Set the name of the state vector channel. This channel will be used to control the flow of data via the on/off bits. Can be given multiple times as --channel-name=IFO=CHANNEL-NAME --dq-channel-name=name Set the name of the data quality channel. This channel will be used to control the flow of data via the on/off bits. Can be given multiple times as --channel-name=IFO=CHANNEL-NAME --shared-memory-partition=name Set the name of the shared memory partition for a given instrument. Can be given multiple times as --shared-memory-partition=IFO=PARTITION-NAME --shared-memory-dir=name Set the name of the shared memory directory for a given instrument. Can be given multiple times as --shared-memory-dir=IFO=DIR-NAME --shared-memory-assumed-duration=SHARED_MEMORY_ASSUMED_DURATION Set the assumed span of files in seconds. Default = 4. --shared-memory-block-size=SHARED_MEMORY_BLOCK_SIZE Set the byte size to read per buffer. Default = 4096. --frame-segments-file=filename Set the name of the LIGO light-weight XML file from which to load frame segments. Optional iff --data- source=frames --frame-segments-name=name Set the name of the segments to extract from the segment tables. Required iff --frame-segments-file is given --state-vector-on-bits=bits Set the state vector on bits to process (optional). The default is 0x7 for all detectors. Override with IFO=bits can be given multiple times. Only currently has meaning for online (lvshm) data. --state-vector-off-bits=bits Set the state vector off bits to process (optional). The default is 0x160 for all detectors. Override with IFO=bits can be given multiple times. Only currently has meaning for online (lvshm) data. --dq-vector-on-bits=bits Set the DQ vector on bits to process (optional). The default is 0x7 for all detectors. Override with IFO=bits can be given multiple times. Only currently has meaning for online (lvshm) data. --dq-vector-off-bits=bits Set the DQ vector off bits to process (optional). The default is 0x160 for all detectors. Override with IFO=bits can be given multiple times. Only currently has meaning for online (lvshm) data.