Testing¶
Unit tests¶
This package has a unit test suite that is set up based on the Astropy Testing Guidelines. The unit tests are run automatically on every git push using GitLab Continuous Integration (CI). See the repository’s .gitlab-ci.yml file for configuration details.
You can also run the unit tests manually by running these commands in the source directory:
$ pip install -e .[test]
$ pytest
There are many options available to adjust what tests are run or how test results are reported; see Astropy’s documentation on running tests for details.
Unit tests in ligo.skymap come in three forms:
Test modules: Generally, each Python subpackage of ligo.skymap has a
tests
directory containing Pytest-style unit tests. For example, unit tests forligo.skymap.plot
are in ligo/skymap/plot/tests.Doctests: The documentation for many modules contains inline examples with expected console output called doctests. When you run the test suite, it checks that the output matches. An example of a module with many doctests is
ligo.skymap.distance
(see source).C unit tests: The critical sections of C code have unit tests that are written in C using GSL’s test framework (see gsl_test.h). These tests are defined in the functions
bayestar_test()
in src/bayestar_sky_map.c andcubic_interp_test()
in src/cubic_interp_test.c.
Coverage analysis¶
The CI pipeline does code coverage analysis when it runs the unit tests. See the coverage report for the most recent build.
Acceptance tests¶
There is a suite of weekly acceptance tests for BAYESTAR that check that the code reproduces localizations for past gravitational-wave events including GW170814 and GW170817 as well as populations of simulated events.
Review¶
This package comprises an analysis pipeline of the Compact Binary Coalescence (CBC) Working Group of the LIGO Scientific Collaboration (LSC). As such, it is required to undergo an LSC scientific review (private wiki link). Minutes of ligo.skymap’s LSC review team are kept in the project’s private wiki.
Scope¶
The essential science capabilities that undergo LSC review are:
Rapid CBC localization (BAYESTAR)
Post-processing of posterior samples to 3D sky maps (ligo-skymap-from-samples)
High-level 2D and 3D localization scripts (ligo-skymap-plot, ligo-skymap-plot-volume)
Roles¶
All review tests (the unit tests and acceptance tests described above) should be automated, take no more than 2 hours to run, and indicate success or failure in a self-evident way. The task of the developers is to create and maintain the automated tests. The task of the reviewers is to provide oversight to ensure that the tests are necessary and sufficient to cover the scientific functionality that is under review.
Changes¶
ligo.skymap is a stable, mature package. Most changes are conservative and maintenance-oriented. The developers will not usually contact the reviewers about these kinds of changes before merging into the main branch. Examples of these kinds of change are adjustments to track API changes in Python, Numpy, and Astropy.
However, the developers will flag potential changes that might need extra scrutiny because they could science results by adding the Review label to merge requests and requiring an approval from a reviewer before merging. Examples of these changes are extracting new parameters from BAYESTAR (e.g. inclination angles) or making significant changes to algorithm inner loops that could affect floating point accuracy.
Releases¶
All stable releases (versions that are triples of numbers of the form
1.2.3
) of ligo.skymap have been approved by the review team. We create one
or more release candidates (versions that are of the form 1.2.3rcN
for some
number N
) until the latest release candidate satisfies all of the tests and
is verbally approved by the review team. Then we do a stable release. The
review team indicates its formal assent to the release by approving the
corresponding ticket in the LSC
Software Change Control Board (SCCB) issue tracker (private link).