PURIFY

Description

PURIFY is an open-source collection of routines written in C++ available under the license below. It implements different tools and high-level to perform radio interferometric imaging, i.e. to recover images from the Fourier measurements taken by radio interferometric telescopes.

PURIFY leverages recent developments in the field of compressive sensing and convex optimization. Low-level functionality to solve the resulting convex optimisation is factored into the open-source companion code, SOPT, also written by the authors of PURIFY. For further background please see the reference section.

This documentation outlines the necessary and optional dependencies upon which PURIFY should be built, before describing installation, testing and usage details. Contributors, references and license information then follows.

Dependencies installation

PURIFY is written in C++11. Required software and libraries, and their minimum supported versions, are listed below. The build system will attempt to automatically download and build the automatically included libraries. (an internet connection is required for this).

C++ dependencies:

• CMake v3.5.1 A free software that allows cross-platform compilation
• GCC v7.3.0 GNU compiler for C++
• OpenMP v4.8.4 - Optional - Speeds up some of the operations.
• MPI v3.1.1 - Optional - Parallelisation paradigm to speed up operations.
• astro-informatics/sopt v4.1.0: Sparse Optimization Compressed Sensing library.
• Boost v1.78.0: A set of free peer-reviewed portable C++ libraries.
• fftw3 v3.3.9: Fastest Fourier Transform in the West.
• Eigen3 v3.3.7: Modern C++ linear algebra.
• tiff v4.0.9: Tag Image File Format library.
• cfitsio: v4.0.0: Library of C and Fortran subroutines for reading and writing data files in FITS (Flexible Image Transport System) data format.
• yaml-cpp v0.6.3: YAML parser and emitter in C++.
• casacore - Optional - Needed to interface with measurement
• ONNXruntime v1.17.1 - Optional - a cross-platform runtime engine based on the Open Neural Network eXchange format. sets.
• Catch2 v2.13.9: Optional - A C++ unit-testing framework only needed for testing.
• google/benchmark v1.6.0: Optional - A C++ micro-benchmarking framework only needed for benchmarks.

For examples on how to install dependencies on Ubuntu and MacOS, see the cmake.yml.

Installing and building PURIFY

If the dependencies are already available on your system, you can also install PURIFY manually like so

cd /path/to/code
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=${PWD}/../local
make -j
make -j install

On MacOS, you can also install most of the dependencies with Homebrew e.g.

brew install boost fftw  eigen yaml-cpp catch2 [onnxruntime]

Machine-learning models

The SOPT library includes an interface to ONNXrt for using trained models as priors in the Forward-Backward optimization algorithm. To build PURIFY with ONNXrt capability, you need to enable ONNXrt support also in SOPT using the onnxrt option when running the cmake command.

Testing

To check everything went all right, run the test suite:

cd /path/to/purify/build
ctest .

Usage

The main purify executable lives either in the build directory or in the in the bin subdirectory of the installation directory. purify has one required argument, it a string for the file path of the config file containing the settings.

purify path/to/config.yaml.

A template with a description of the settings is included in the data/config directory. When purify runs a directory will be created, and the output images will be saved and time-stamped. Additionally, a config file with the settings used will be saved and time-stamped, helping for reproducibility and book-keeping.

Uncertainty Quantification

Bayesian hypothesis testing may be performed with the purify_UQ application, which can be located in the build directory.

This application takes a config yaml file with the following parameters:

• confidence_interval or alpha. (alpha = 1 - confidence interval.))
• measurements_path: path to measurements data (.vis file)
• reference_image_path: path to reference image i.e. output from purify run. (.fits files)
• surrogate_image_path: path to surrogate image i.e. doctored image with blurring or structural change that you want to test. (.fits file)
• sigma: standard deviation for Gaussian likelihood.
• gamma: multiplicative factor for prior.
• purify_config: path to purify config used to generate the reference image. This should be used if available in order to ensure consistency of things like measurement and wavelet operators.

You can then run the uncertainty quantification with the command:

purify_UQ <path to UQ_config yaml>

The application will report the value of the objective function for each image, the threshold value calculated from the reference image, and whether the surrogate image is ruled out or not.

Presently this is designed to work for the unconstrained problem where:

• The negative log-likelihood is a scaled L2-norm i.e. $ \frac{1}{2 \sigma^2} \sum (y_i - \Phi x_i)$ for some data $y$, image $x$, and measurement operator $\Phi$. (Equivalent to indepdendent multivariate Gaussian likelihood.)
• The negative log prior is a scaled L1-norm in wavelet space i.e. $\gamma \sum (\Psi^\dag x)_i$ for some image $x$ and wavelet operator $\Psi$.
• The objective function is the sum of these two terms.

Docker

Debugging the CI workflow with tmate

The CI workflow has a manual dispatch trigger which allows you to log into the job while it's running. You can trigger it in actions. Run the workflow and set debug_enabled=true to enable the tmate step in the CI workflow. Once the workflow is running, open the job in actions. You should see it printing out a line with a ssh command. Run it in terminal to log into the GitHub Actions runner.

Docker

A Dockerfile is available on DockerHub. We are currently not maintaining it, and cannot guarantee it is up to date. Use the below documentation at your own risk.

If you want to use Docker instead, you can build an image using the Dockerfile available in the repository or pulling it from DockerHub.

docker build -t purify .

or

docker pull uclrits/purify

Then to use it, you should mount the directory with your data and config files to /mydata in the container. To run the container and mount the directory is with:

docker run -it --name purify -v /full/path/to/data:/mydata uclrits/purify

That will start a shell inside the container in the /mydata directory where you can see all the files from your /full/path/to/data. There you can run purify as shown above.`

Contributors

Check the contributors page (github).

References and citation

If you use PURIFY for work that results in publication, please reference the webpage and our related academic papers:

1. L. Pratley, et al. "Distributed convex optimization for Radio Interferometry with PURIFY". Link will be here soon!
2. L. Pratley, M. Johnston-Hollitt, J. D. McEwen, "A fast and exact w-stacking and w-projection hybrid algorithm for wide-field interferometric imaging". Submitted to ApJ arXiv:1807.09239
3. L. Pratley, J. D. McEwen, M. d'Avezac, R. E. Carrillo, A. Onose, Y. Wiaux. "Robust sparse image reconstruction of radio interferometric observations with PURIFY". Accepted (2016) arxiv:1610.02400
4. A. Onose, R. E. Carrillo, A. Repetti, J. D. McEwen, J.-P. Thiran, J.-C. Pesquet, and Y. Wiaux. "Scalable splitting algorithms for big-data interferometric imaging in the SKA era" Mon. Not. Roy. Astron. Soc. 462(4):4314-4335 (2016) arXiv:1601.04026
5. R. E. Carrillo, J. D. McEwen and Y. Wiaux. "PURIFY: a new approach to radio-interferometric imaging". Mon. Not. Roy. Astron. Soc. 439(4):3591-3604 (2014) arXiv:1307.4370

License

PURIFY Copyright (C) 2013-2019

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details (LICENSE.txt).

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

Webpage

• Documentation
• Repository

Support

For any questions or comments, feel free to contact Jason McEwen, or add an issue to the issue tracker.

Notes

The code is given for educational purpose. The code is in beta and still under development.