Data Products

The different data product types in CASDA are described below. The Data Access Portal (DAP) User Interface allows the user to easily see the type and filter for desired data products.

Image or Cube Types 

Each image or image cube is stored in CASDA in FITS format.  We describe them here with an example filename, separating out the types into categories. 

Image types – Continuum, two-dimensional 

  • cont_restored_t0 – The primary image type. This is the restored total-intensity (or Taylor-0) image (summed over the entire bandwidth being processed). Example: image.i.SB1234.cont.taylor.0.restored.fits 
  • cont_restored_t1 – The “Taylor-1” image, produced in multi-frequency synthesis. This shows the spectral index at each point multiplied by the total intensity. Used to derive spectral indices for catalogued components. Example: image.i.SB1234.cont.taylor.1.restored.fits 
  • cont_residual_t0 – The residual flux remaining from deconvolving & cleaning the total-intensity image (ie. shows the un-cleaned emission). Example: residual.i.SB1234.cont.taylor.0.fits 
  • cont_residual_t1 – The residual flux remaining from deconvolving & cleaning the Taylor-1 image. Only produced by multi-frequency synthesis. Example: residual.i.SB1234.cont.taylor.1.fits 
  • cont_cleanmodel_t0 – The image of the deconvolved model resulting from cleaning the total intensity image. Example: image.i.SB1234.cont.taylor.0.fits 
  • cont_cleanmodel_t1 – The image of the deconvolved model resulting from cleaning the Taylor-1 image. Example: image.i.SB1234.cont.taylor.1.fits 
  • cont_weight_t0 – The relative sensitivity across the field. This typically is a product of the linear-mosaicking approach, and shows the effect of the primary beam attenuation, coupled with the number of visibilities contributing to a given PAF beam. This version goes with the total-intensity Taylor-0 image. Example: weights.i.SB1234.cont.taylor.0.fits 
  • cont_weight_t1 – The relative sensitivity image that goes with the Taylor-1 image. Example: weights.i.SB1234.cont.taylor.1.fits 
  • cont_components_t0 – A map of fitted components, that are identified as part of the cataloguing process and fitted to the total intensity image. These are Gaussian components that appear in the component catalogue. Example: componentMap_image.i.SB1234.cont.taylor.0.restored.fits 
  • cont_fitresidual_t0 – A map of the residual emission after the component map is subtracted from the total intensity image. Example: componentResidual_image.i.SB1234.cont.taylor.0.restored.fits 
  • cont_noise_t0 – A map of the noise in the total-intensity image as a function of position across the field. This is typically created during the source-finding process. Example: noiseMap.image.i.SB1234.cont.taylor.0.restored.fits 

Image types – Continuum, three-dimensional 

  • cont_restored_3d – The primary continuum cube type. This is the restored total-intensity image in each coarse (continuum) channel. Example: image.restored.i.SB1234.contcube.fits 
  • cont_residual_3d – The residual flux remaining in each coarse channel after deconvolution. Example: residual.i.SB1234.contcube.fits 
  • cont_cleanmodel_3d – The image of the deconvolved model resulting from cleaning each coarse channel. Example: image.i.SB1234.contcube.fits 
  • cont_weight_3d – The relative sensitivity in each coarse channel. Example: weights.i.SB1234.contcube.fits 

Image types – Spectral-line, three-dimensional 

  • spectral_restored_3d – The primary spectral-line cube type. This is the restored total-intensity image in each fine spectral channel. This cube may have had the continuum emission removed. Examples: image.restored.i.SB1234.cube.fits  or image.restored.i.SB1234.cube.contsub.fits  
  • spectral_residual_3d – The residual flux remaining in each fine channel after deconvolution. Examples: residual.i.SB1234.cube.fits 
  • spectral_cleanmodel_3d – The image of the deconvolved model resulting from cleaning each fine channel. Examples: image.i.SB1234.cube.fits 
  • spectral_weight_3d – The relative sensitivity in each fine channel. Examples: weights.i.SB1234.cube.fits 
  • spectral_restored_mom0 – A two-dimensional image showing the moment-0 (or summed intensity over a frequency range) image of a spectral cube. 
  • spectral_restored_mom1 – A two-dimensional image showing the moment-1 (or average spectral value) image of a spectral cube. 

Image file names 

The image products produced by the ASKAPsoft pipeline conform to particular patterns, that allow one to discern the type of file that it is. 

The general form of an image name is {prefix}{imagekind}.{polarisation}.SBxxxxx.{datatype}.{suffix}.fits where the individual elements are: 

  • {prefix} – this is used for derived images that are created after the imaging (by the source-finding software, for instance). 
  • {imagekind} – the kind of image, usually one of “image”, “residual”, “weights”. For cubes, this could also be “image.restored” (but continuum images have the “restored” tag in the suffix). 
  • {polarisation} – the lower-case version of the Stokes parameter: usually one of “i”, “q”, “u” or “v” 
  • SBxxxxx – the scheduling block (SB) ID number 
  • {datatype} – this indicates whether it is a continuum image (“cont”), a continuum cube (coarse-resolution, typically 1MHz channels – “contcube”), or a spectral cube (fine resolution, usually 18.5kHz channels or finer – “cube”). 
  • {suffix} – this can indicate things like the multi-frequency synthesis information (“taylor.0”), whether it is a restored continuum image (“restored”), whether the cube has been continuum-subtracted (“contsub”). There are several types of restored continuum images that may be presented
    • “restored.conv.fits” – these are mosaics that have had all constituent beams convolved to a common resolution prior to mosaicking. The PSF in the header will accurately reflect the resolution across the field and fluxes will be accurate.
    • “restored.raw.fits” – a mosaic created from beam images prior to the convolution. The resolution will be better, but beam-dependent, and the PSF in the header will not necessarily reflect the resolution at an arbitrary point in the image (leading to potential flux errors when making measurements).
    • “highres.restored.conv.fits” or “alt.restored.conv.fits” – indicates an alternative preconditioning parameterisation was used to restore the image (in addition to the regular preconditioning). The name varies according to processing template, but this is often done at a lower robustness value, hence the indication of high resolution (“highres”).

Catalogue Types

There are different catalogues created by Selavy, the ASKAP source-finder. These catalogues are described in the ASKAPsoft user documentation for Selavy at https://www.atnf.csiro.au/computing/software/askapsoft/sdp/docs/current/analysis/index.html.  

  • Continuum Island – a catalogue of the “islands” in the  total-intensity image. An island is a group of contiguous pixels that are above some detection threshold.  
  • Continuum Component – a catalogue of the components that make up each island. A component is a two-dimensional Gaussian, parameterised by a location, flux, size and orientation. Each island has some number of components fitted to it, so there is a one-to-many relationship between the island and component catalogues. 
  • Polarisation Component – a catalogue of polarisation and rotation measure properties for each component in the component catalogue.  
  • Spectral line Emission – a catalogue of emission-line sources extracted from three-dimensional source-finding on the total-intensity spectral-line cube. 

Other data types

Other data types in CASDA 

Measurement sets 

The visibility data stored in CASDA is in the form of Measurement Sets, that have been packaged as tar files. A Measurement Set (MS) has a directory structure, and will contain a series of CASA-format tables that describe a lot of the observation metadata, as well as the main table with the visibility data. Each MS will have data just for a single PAF beam. 

To supplement the MS metadata, we have added an additional directory, called “ASKAP_METADATA”, containing the metadata files used by the pipeline processing.  

Evaluation files 

The evaluation files are a collection of ancillary data that either provide information about the quality of the data or information about the processing. These are typically available via download links from the information tab for each image product. There are several distinct types: 

  • Calibration-metadata-processing-logs – this is a tar file containing a large amount of information from the processing. Its filename will have a timestamp (of when the processing was begun).  This contains: 
    • Calibration tables, for bandpass, self-calibration, and leakage (if done). These tables are CASA-format, that are understood by the ASKAPsoft processing software.  
    • A diagnostics directory, containing plots and flagging summaries for each of the MSs 
    • The pipeline metadata directory (that is also copied to the MSs, as mentioned above) 
    • Tarred directories of logs and parameter set inputs for each of the processing jobs 
  • Validation results – these are tar files containing the directory of science validation results. There may be one or more set of results corresponding to validation scripts developed by Survey Science Teams:
    • EMU – cross matches the component catalogue with RACS-low, and assesses position & flux accuracy
    • POSSUM – detailed validation looking at the polarisation spectra, including assessing wide-field leakage correction
    • GASKAP-HI – quality validation of the visibility data and calibration
    • WALLABY, DINGO, FLASH – closely-related validation reports looking at the spectral properties of noise and other statistics
  • Extracted spectra – when 1D spectra are extracted from the larger spectral cubes, these are combined together into tar files. Each file will contain spectra of a common type (source spectra, noise spectra, Faraday spectra for the case of RM synthesis).
  • An “encaps” file – an “encapsulation” of the pipeline configuration files used and the continuum validation quality metrics (as an XML file).