CASDA Release Notes

CASDA v1.20:  21 Sep 2022

Search

  • Add new CASDA Cutout Service page for producing a cutout from curated survey data
  • Observation search – fix error when creating cutout from a cone search result
  • Observation search – cutout service now uses RA, Dec and cutout radius, instead of box corners

Data Access

  • Add support for Simple Image Access v1
  • Update checksum file names to match data file names

VO Tools

  • Add support for Simple Image Access v1

Deposit

  • Provide extra metadata to Datacite when registering DOIs
  • Move from /group to /askapbuffer for observational data deposits
  • Add support for unicode characters in derived data deposits
  • Remove use of NGAS

Server Migration

  • Move database to new Nimbus virtual machines
  • Move Data Access and VO Tools to new Nimbus virtual machines

CASDA v1.19:  2 June 2022

Search

  • Show derived images on the images tab
  • Correct search by velocity

Data Access

  • Fix for CASDA incorrectly showing as unavailable after outage

Deposit

  • Stability improvements to the derived data deposit process
  • Add support for linked files in derived data
  • Carry files forward from the earlier version when a derived collection is updated
  • Add support for NaN values in derived catalogues
  • Verify files in observation.xml are not directories

S3 Data Migration

  • Completion of migration of all data to the Acacia object storage system at Pawsey

CASDA v1.18:  10 February 2022

Search:

  • Show catalogue filenames in search result (delayed for DAP 2.34 release)

Data Access

  • Add rejected events to the observation event list showing when an entire observation is not released
  • Hide rejected data from SSA
  • Bug fixes for accessing data in object storage

Deposit

  • Make the level 7 deposit process more robust (delayed for DAP 2.34 release)
  • Fix issue where updated observation events were not being recorded for some redeposits
  • Populate start and stop time fields (t_min and t_max) in obscore for image cubes

S3 Data Migration

  • Correct indexing of data after migration
  • Add support for multiple servers to migrate data

VO Tools:

  • Update log4j to v2.17.1

CASDA v1.17:  16 December 2021

VO Tools:

  • Register the RACS catalogues and their SCS interfaces in our VO registry entries
  • Datalink – Correct handling of future release dates
  • Datalink – Ensure Astroquery uses public services, not Pawsey only ones
  • Correct handling of column metadata for columns with a different name in TAP vs database
  • Added a CASDA VO Tools server header
  • Improve TAP standards compliance

Deposit

  • Complete support for depositing data to object storage

Data Access

  • Complete support for accessing data in object storage
  • Tidy up validation report URLs
  • Skymap – correct previews for data with spectral axes

S3 Data Migration

  • Add support for migrating low priority data from Banksia to Acacia
  • Add support for increased storage for high-priority data for initial transfer to Acacia

CASDA v1.16:  21 October 2021

Search:

  • Make the cut-out service more obvious in the new UI.
  • Include directions on creating OPAL accounts.
  • Quick access to larger images from search results.
  • Mark files that are in the process of migration to S3 as temporarily unavailable for download.
  • Improve paging in cone search results

Deposit:

  • Enable multiple migration jobs to run in parallel.
  • Level 7 – Update release dates when a metadata only update is published.

CASDA v1.15:  6 September 2021

Deposit:

  • Implement migration process to migrate all CASDA data to Pawsey’s new object storage system Acacia
  • Commence preparing high priority data for migration
  • Allow unicodeChar type in metrics files
  • Fix issue where duplicate encapsulation files could be created
  • Fix issue where redeposits would fail if catalogues from the original deposit were no longer on the lustre filesystem
  • Preparation for depositing files into object storage

Data Access:

  • Preparation for access to files in object storage

VO Tools:

  • Correct validation errors: E_TSTD – TAP_SCHEMA column not flagged Std

CASDA v1.14:  1 July 2021

Search:

  • Add commensal project search option

Deposit:

  • Support for recording commensal projects for data products in ‘formally commensal’ observations
  • Add support for deposit of large derived catalogues
  • Add partial release option to the release task – this will reject bad data and release all other data

Data Access:

  • Add RACS as the default layer to the skymap display
  • Enable selected DOI to be easily copied when on the data tab of a CASDA collection
  • Correct zoom on skymap object preview

VO Tools:

  • Correct SIA2 capabilities endpoint missing accessURL

CASDA v1.13: 12 Nov 2020

Search:

  • Correct issue where data products were still showing as unreleased after their embargo period had ended
  • Only show published derived data products in search results

Deposit:

  • Move validate and release tasks for observations to the new user interface
  • Correct issue where older catalogues would fail to be released
  • Add new background image types
  • Link catalogue to their image cube by file name only, ignore the path listed
  • Provide script to allow bulk validation and release of scheduling blocks

Data Access:

  • Provide a new observation event list and query facility
  • Ensure images and cubes for derived collections are shown on the images tab only

VO Tools:

  • Correct output of large calculated numeric values

CASDA v1.12:  9 July 2020

Search:

  • Highlight primary image data products in search results
  • Add derived catalogues and data products to the skymap search
  • Allow multiple facet values to be selected to filter search results
  • Hide rejected data products by default
  • Access validation reports from data product info page
  • Allow both released and unreleased data to be displayed in a search
  • Accept decimal and sexagesimal coordinates in all entry locations

Deposit:

  • Public web hosting of validation reports produced by the ASKAPSoft pipeline
  • Allow derived (level 7) collections to contain catalogues, images and spectral data products in the one collection
  • Speed up deposit of many small files
  • Only add catalogues to global catalogues when released
  • Handle deposits with an unknown project code more gracefully
  • Add lower limit for RESTFREQ from ASKAPsoft
  • Allow plus signs in file names
  • Hide already rejected data products on validation and release pages
  • Correct interaction of observation metadata refresh and redeposit
  • Correct problem handling spectra files with very small footprints

Data Access:

  • Extend CASDA astroquery module to support staging and download of data products
  • Add channel selection for SODA
  • Download original catalogues rather than regenerate from global catalogues
  • Update to Montage v6 for cutout processing
  • Show the file name on the image generation page
  • Improve error messages for SODA requests
  • Correct issue preventing old level 7 data products from being downloaded via the UI or sync SODA download requests

VO Tools:

  • Correct s_resolution values for image cubes in obscore
  • Correct error in tap_tables config preventing new installations

CASDA v1.11a:  30 August 2019

Search:

  • ASVO common authentication – can now login to DAP using AAO Data Central Lens account
  • Restore coverage map to new user interface
  • Fix problems viewing previews of sources in the tucana image.

Data Access:

  • Astroquery module for querying ASKAP data
  • cutouts_channels.py script in CASDA github repository, which divides whole ASKAP cube in to channels before downloading the sub-cubes
  • Resolved issue with job queue stalling

CASDA v1.11:  30 July 2019

Search:

  • Include visibilities in cone search results
  • Correct issue with rest frequency selection
  • Fix error revising cone search criteria

Deposit:

  • Allow project admins to manage membership of their project teams
  • Use original names for files on the validation pages
  • Hide already released data products on validation and release pages

Data Access:

  • Use original names for downloaded data files
  • Increased the download limit per job to 500GB
  • Allow panning on cutout selection map
  • Allow cutouts from cubes crossing 0 RA
  • Allow cutouts of entire planes of a cube
  • Snap selected cutout region to bounds of cube
  • Evaluation files for download now separate into i) ASKAP pipeline logs and calibration table data and ii) quality assessment pipeline output.
  • Update UCD for SODA ID parameter (SODA v1.0 erratum #1)

VO Tools:

  • Add support for ObsCore v1.1
  • Provide more flexibility for the locations of the VO tools config
  • Add actual file name to obscore
  • Add support for embargoed catalogues
  • Encrypt admin password when stored