Container Development Environment

The container development workflow consists of a few key points:

  • Build tools provided by and used within a writable gstlal container.

  • Editor/git used in or outside of the container as desired.

  • Applications are run in the development container.

The benefits of developing in a writable container:

  • Your builds do not depend on the software installed on the system, you don’t have to worry about behavior changes due to system package updates.

  • Your build environment is the same as that of everyone else using the same base container. This makes for easier collaboration.

  • Others can run your containers and get the same results. You don’t have to worry about environment mis-matches.

Create a writable container

The base of a development environment is a gstlal container. It is typical to start with the current master build. However, you can use the build tools to overwite the install in the container so the choice of branch in your gstlal repository matters more than the container that you start with. The job of the container is to provide a well-defined set of dependencies.

    singularity build --sandbox --fix-perms CONTAINER_NAME docker://containers.ligo.org/lscsoft/gstlal:master

This will creat a directory named CONTAINER_NAME. That directory is a singularity container.

Check out gstlal

In a directory of your choice, under your home directory, run:

    git clone https://git.ligo.org/lscsoft/gstlal  DIRNAME

This will create a git directory named DIRNAME which is referred to in the following as your “gstlal dir”. The gstlal dir contains several directories that contain components that can be built independently (e.g., gstlal, gstlal-inspiral, gstlal-ugly, …).

A common practice is to run the clone command in the CONTAINER_NAME directory and use src as DIRNAME. In this case, when you run your container, your source will be available in the directory /src.

Develop

Edit and make changes under your gstlal dir using editors and git outside of the container (or inside if you prefer).

Build a component

To build a component:

  1. cd to your gstlal directory

  2. Run your container:

   singularity run --writable -B $TMPDIR CONTAINER_NAME /bin/bash

  1. cd to the component directory under your gstlal dir.

  2. Initialize the build system for your component. You only need to do this once per container per component directory:

	./00init.sh
	./configure --prefix=/usr --libdir=/usr/lib64

The arguments to configure are required so that you overwrite the build of gstlal in your container.

Some components have dependencies on others. You should build GstLAL components in the following order:

  1. gstlal

  2. gstlal-ugly

  3. gstlal-inspiral, gstlal-burst, gstlal-calibrarion (in any order)

For example, if you want to build gstlal-ugly, you should build gstlal first.

  1. Run make and make install

     make
     make install

Note that the container is writable, so your installs will persist after you exit the container and run it again.

Run your code

You can run your code in the following ways:

  1. Run your container using singularity and issue commands interactively “inside the container”:

    singularity run --writable -B $TMPDIR PATH_TO_CONTAINER  /bin/bash
/bin/gstlal_reference_psd  --channel-name=H1=foo --data-source=white  --write-psd=out.psd.xml --gps-start-time=1185493488 --gps-end-time=1185493788

  2. Use singularity exec and give your command on the singularity command line:

    singularity exec --writable -B $TMPDIR PATH_TO_CONTAINER  /bin/gstlal_reference_psd  --channel-name=H1=foo --data-source=white  --write-psd=out.psd.xml --gps-start-time=1185493488 --gps-end-time=1185493788

  3. Use your container in a new or existing container-based gstlal workflow on a cluster with a shared filesystem where your container resides. For example, you can run on the CIT cluster or on the PSU cluster, but not via the OSG (you can run your container as long as your container is available on the shared filesystem of the cluster where you want to run). In order to run your code on the OSG, you would have to arrange to have your container published to cvmfs.