summarycompare
The summarycompare executable is designed to compare multiple result files. This can be multiple PESummary metafiles or simply two dat files containing posterior samples.
To see help for this executable please run:
$ summarycompare --help
2024-05-14 08:02:17 PESummary WARNING : Unable to install 'pycbc'. You will not be able to use some of the inbuilt functions.
usage: summarycompare [-h] [-s SAMPLES [SAMPLES ...]] [-w DIR] [-v]
[--properties_to_compare {posterior_samples,config,priors,psds} [{posterior_samples,config,priors,psds} ...]]
[--generate_comparison_page]
This executable is used to compare multiple files
optional arguments:
-h, --help show this help message and exit
-s SAMPLES [SAMPLES ...], --samples SAMPLES [SAMPLES ...]
Path to posterior samples file(s). See documentation
for allowed formats. If path is on a remote server,
add username and servername in the form
{username}@{servername}:{path}. If path is on a public
webpage, ensure the path starts with https://. You may
also pass a string such as posterior_samples*.dat and
all matching files will be used
-w DIR, --webdir DIR make page and plots in DIR
-v, --verbose print useful information for debugging purposes
--properties_to_compare {posterior_samples,config,priors,psds} [{posterior_samples,config,priors,psds} ...]
list of properties you wish to compare between the
files. Default posterior_samples
--generate_comparison_page
Generate a comparison page to compare contents
Further details
The sumamrycompare executable loads in each result file with the pesummary.io module and compares the properties. If the properties are a dictionary, we recursively search through the entries, until a string, float or numpy array can be compared between the two result files.
We print the differences to stdout through logger. If there are no differences between the result files, no information is printed to stdout. However, this can be modified by passing the -v/–verbose command line argument.
Examples
Below we show an example where we compare the same metafiles:
$ summarycompare --samples posterior_samples.h5 posterior_samples.h5 \
--properties_to_compare posterior_samples config priors
2020-06-08 13:25:34 PESummary INFO : Command line arguments: Namespace(compare=['posterior_samples', 'config', 'priors'], samples=['webpage/samples/posterior_samples.h5', 'webpage/samples/posterior_samples.h5'], verbose=False)
$
As expected, nothing is returned as there are no differences. If we ran with the verbose option:
$ summarycompare --samples posterior_samples.h5 posterior_samples.h5 \
--properties_to_compare posterior_samples config priors \
--verbose
2020-06-08 13:26:51 PESummary INFO : Command line arguments: Namespace(compare=['posterior_samples', 'config', 'priors'], samples=['webpage/samples/posterior_samples.h5', 'webpage/samples/posterior_samples.h5'], verbose=True)
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_matched_filter_abs_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_matched_filter_snr_angle'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_optimal_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_amp_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/H1_spcal_phase_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_matched_filter_abs_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_matched_filter_snr_angle'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_optimal_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_amp_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/L1_spcal_phase_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_matched_filter_abs_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_matched_filter_snr_angle'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_optimal_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_amp_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_2'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_3'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_4'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_5'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_6'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_7'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_8'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/V1_spcal_phase_9'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/azimuth'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/deltalogl'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/log_likelihood'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/loglH1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/loglL1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/loglV1'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/logpost'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/log_prior'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/network_matched_filter_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/nullLogL'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/network_optimal_snr'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/phase'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/phi_12'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/phi_jl'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/mass_ratio'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/t0'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/temperature'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/geocent_time'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/nLocalTemps'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/randomSeed'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/ra'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/dec'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/luminosity_distance'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/psi'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/chirp_mass'
2020-06-08 13:27:00 PESummary DEBUG : The result files match for entry: 'samples_dict/one/a_1'
.
.
.
We now see alot more information and can see which entries have been compared.
Of course, this does not just have to be used to compare PESummary metafiles. We can also compare two files containing only the posterior samples, for example:
>>> import numpy as np
>>> from pesummary.io import write
>>> parameters = ["a", "b", "c", "d"]
>>> data = np.random.random([100, 4])
>>> write(parameters, data, file_format="dat", filename="example1.dat")
>>> parameters2 = ["a", "b", "c", "d", "e"]
>>> data2 = np.random.random([100, 5])
>>> write(parameters2, data2, file_format="json", filename="example2.json")
$ summarycompare --samples example1.dat example2.json \
--properties_to_compare posterior_samples -v
2020-06-08 13:36:45 PESummary INFO : Command line arguments: Namespace(compare=['posterior_samples'], samples=['example1.dat', 'example2.json'], verbose=False)
2020-06-08 13:36:47 PESummary WARNING : Failed to find 'log_likelihood' in result file. Setting every sample to have log_likelihood 0
2020-06-08 13:36:47 PESummary INFO : Failed to read in example2.json with the <bound method PESummary.load_file of <class 'pesummary.gw.file.formats.pesummary.PESummaryDeprecated'>> class because __init__() got an unexpected keyword argument 'disable_prior_conversion'
2020-06-08 13:36:47 PESummary WARNING : Using the default load because example2.json failed the following checks: is_bilby_json_file, is_pesummary_json_file, is_pesummary_json_file_deprecated
2020-06-08 13:36:47 PESummary WARNING : Failed to find 'log_likelihood' in result file. Setting every sample to have log_likelihood 0
2020-06-08 13:36:47 PESummary INFO : The result files differ for the following entry: 'samples_dict/a'. The maximum difference is: 0.863436584455099
2020-06-08 13:36:47 PESummary INFO : The result files differ for the following entry: 'samples_dict/b'. The maximum difference is: 0.866345243653647
2020-06-08 13:36:47 PESummary INFO : The result files differ for the following entry: 'samples_dict/c'. The maximum difference is: 0.892060295600003
2020-06-08 13:36:47 PESummary INFO : The result files differ for the following entry: 'samples_dict/d'. The maximum difference is: 0.8092611957933932
2020-06-08 13:37:31 PESummary DEBUG : The result files match for entry: 'samples_dict/log_likelihood'