CASDA storage migration

As part of a $70 million capital refresh project, the Pawsey Supercomputing Centre is in the process of installing one of the largest research-focused object storage systems in the world: 130 PB of online and offline storage.

The new online object storage system, Acacia, is now being made available to researchers, with CASDA one of the first users of the system. Over the last year careful planning and significant development has gone into ensuring data integrity and making this transition as seamless as possible.

We are pleased to announce that we have migrated ~600 TB of high priority data to Acacia in the last week and are now serving this data from CASDA. This consists of catalogues, primary images/cubes, spectra, value-added user deposits, and ancillary data products (e.g. weight and noise maps) requested by the ASKAP Survey Science Teams. Deposits are now also to Acacia, so all newly deposited data is also available.

The remaining low-priority data (visibilities and ancillary images), almost 1 PB of data, will be transferred once the new offline/tape system, Banksia, is in production. This secondary data migration is expected to start on Mar 21 and take 4 to 6 weeks.

The new Acacia object storage will result in better performance for users, as files can be served immediately instead of being queued to come off tape. We expect further improvements in the future as we look to take full advantage of object storage.

If you have any queries about data availability please contact Minh Huynh (CASDA lead).

The back of Acacia, Pawsey’s new object storage system. Credit: Karina Nunez, Pawsey Supercomputing Centre.