eSDO 1321: Data Centre Integration Plan

This document can be viewed as a PDF.
Deliverable eSDO-1321
E. Auden
23 August 2005

UK Data Centre

The UK data centre will host some data products from the Helioseismic and Magnetic Imager (HMI) and the Atmospheric Imaging Assembly (AIA). Helioseismologists will be interested in HMI data collected for long periods of time. By contrast, AIA data will cater to solar physicists analysing events such as flares and coronal mass ejections; this audience will primarily be interested in recent data. Low and high level data products from both instruments will be permanently available through the US data centre, so the UK data centre holdings will focus on providing fast access to data of the most interest to solar scientists across the UK.

Archived Data

HMI

The following HMI data products will be archived for the nominal duration of the SDO mission, 2008 - 2014: full resolution line-of-sight magnetograms (1 / 45 seconds), full resolution dopplergrams (1 / 45 seconds), and full resolution filtergrams averaged over a 20 minute period (1 / 20 minutes). Uncompressed storage for these data products is currently estimated at ~33 TB per year, culminating in 198 TB total storage by 2014.

Data Product Instrument Estimated Size Estimated Cadence Storage

AIA

AIA products will be held in a rolling 60 day cache; this will provide solar physicists with data from two most recent solar revolutions. Cached low level data will include low resolution full-disk images (8 per 10 seconds) along with high resolution images of tracked active regions (8 per 10 seconds). Several high level products generated at a much lower cadence will also be cached: thermal maps, DEM measures, irradiance estimates, and magnetic field extrapolations (1 to 10 per day). The storage requirement for a rolling 60 day cache is estimated at 11 TB.

Data Product Instrument Estimated Size Estimated Cadence Storage

Quicklook and Visualization

To aid visualisation and searching of the vast SDO archives, “quicklook” products such as thumbnail images will be generated for all low level HMI and AIA data held in the UK archive. The UK solar community has expressed interest in generating movies of this data; rather than storing pre-canned movies, a web service will generate movies of user-delimited start and end times from archived HMI or AIA thumbnail images. All thumbnails will be archived for the duration of the SDO mission, requiring an addition estimated 10 TB of space per year.

Data Product Instrument Estimated Size Estimated Cadence Storage

Synopsis of Storage Requirements

Year Instrument Storage Facility Cumulative Size
08/2008 - 08/2009 HMI ADS tape archive (permanent) 33 TB
08/2009 - 08/2010 HMI ADS tape archive (permanent) 66 TB
08/2010 - 08/2011 HMI ADS tape archive (permanent) 99 TB
08/2011 - 08/2012 HMI ADS tape archive (permanent) 132 TB
08/2012 - 08/2013 HMI ADS tape archive (permanent) 165 TB
08/2013 - 08/2014 HMI ADS tape archive (permanent) 198 TB
08/2008 - 08/2009 AIA ADS disc cache (60 days) 11 TB
08/2009 - 08/2010 AIA ADS disc cache (60 days) 11 TB
08/2010 - 08/2011 AIA ADS disc cache (60 days) 11 TB
08/2011 - 08/2012 AIA ADS disc cache (60 days) 11 TB
08/2012 - 08/2013 AIA ADS disc cache (60 days) 11 TB
08/2013 - 08/2014 AIA ADS disc cache (60 days) 11 TB
08/2008 - 08/2009 Thumbnails ADS tape archive (permanent) 14 TB
08/2009 - 08/2010 Thumbnails ADS tape archive (permanent) 28 TB
08/2010 - 08/2011 Thumbnails ADS tape archive (permanent) 42 TB
08/2011 - 08/2012 Thumbnails ADS tape archive (permanent) 66 TB
08/2012 - 08/2013 Thumbnails ADS tape archive (permanent) 80 TB
08/2013 - 08/2014 Thumbnails ADS tape archive (permanent) 94 TB

Total storage requirements: 11 TB rolling cache, 47 TB / year permanent storage over 6 years (282 TB at the end of year 6)

-- ElizabethAuden - 23 Aug 2005

Edit | Attach | Watch | Print version | History: r7 | r5 < r4 < r3 < r2 | Backlinks | Raw View | More topic actions...
Topic revision: r3 - 2005-09-29 - ElizabethAuden
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback