Difference: DataCentreIntegration1321 (r4 vs. r3)

eSDO 1321: Data Centre Integration Plan

This document can be viewed as a PDF.
Deliverable eSDO-1321
E. Auden
23 August 2005

UK Data Centre

The UK data centre will host some data products from the Helioseismic and Magnetic Imager (HMI) and the Atmospheric Imaging Assembly (AIA). Helioseismologists will be interested in HMI data collected for long periods of time. By contrast, AIA data will cater to solar physicists analysing events such as flares and coronal mass ejections; this audience will primarily be interested in recent data. Low and high level data products from both instruments will be permanently available through the US data centre, so the UK data centre holdings will focus on providing fast access to data of the most interest to solar scientists across the UK.

Archived Data

HMI

The following HMI data products will be archived for the nominal duration of the SDO mission, 2008 - 2014: full resolution line-of-sight magnetograms (1 / 45 seconds), full resolution dopplergrams (1 / 45 seconds), and full resolution filtergrams averaged over a 20 minute period (1 / 20 minutes). Uncompressed storage for these data products is currently estimated at ~33 TB per year, culminating in 198 TB total storage by 2014.

Data ProductInstrumentEstimated SizeEstimated CadenceStorage

AIA

AIA products will be held in a rolling 60 day cache; this will provide solar physicists with data from two most recent solar revolutions. Cached low level data will include low resolution full-disk images (8 per 10 seconds) along with high resolution images of tracked active regions (8 per 10 seconds). Several high level products generated at a much lower cadence will also be cached: thermal maps, DEM measures, irradiance estimates, and magnetic field extrapolations (1 to 10 per day). The storage requirement for a rolling 60 day cache is estimated at 11 TB.

Data ProductInstrumentEstimated SizeEstimated CadenceStorage

Quicklook and Visualization

To aid visualisation and searching of the vast SDO archives, “quicklook” products such as thumbnail images will be generated for all low level HMI and AIA data held in the UK archive. The UK solar community has expressed interest in generating movies of this data; rather than storing pre-canned movies, a web service will generate movies of user-delimited start and end times from archived HMI or AIA thumbnail images. All thumbnails will be archived for the duration of the SDO mission, requiring an addition estimated 10 TB of space per year.

Data ProductInstrumentEstimated SizeEstimated CadenceStorage

Synopsis of Storage Requirements

YearInstrumentStorage FacilityCumulative Size
08/2008 - 08/2009 HMI ADS tape archive (permanent) 33 TB
08/2009 - 08/2010 HMI ADS tape archive (permanent) 66 TB
08/2010 - 08/2011 HMI ADS tape archive (permanent) 99 TB
08/2011 - 08/2012 HMI ADS tape archive (permanent) 132 TB
08/2012 - 08/2013 HMI ADS tape archive (permanent) 165 TB
08/2013 - 08/2014 HMI ADS tape archive (permanent) 198 TB
08/2008 - 08/2009 AIA ADS disc cache (60 days) 11 TB
08/2009 - 08/2010 AIA ADS disc cache (60 days) 11 TB
08/2010 - 08/2011 AIA ADS disc cache (60 days) 11 TB
08/2011 - 08/2012 AIA ADS disc cache (60 days) 11 TB
08/2012 - 08/2013 AIA ADS disc cache (60 days) 11 TB
08/2013 - 08/2014 AIA ADS disc cache (60 days) 11 TB
08/2008 - 08/2009 Thumbnails ADS tape archive (permanent) 14 TB
08/2009 - 08/2010 Thumbnails ADS tape archive (permanent) 28 TB
08/2010 - 08/2011 Thumbnails ADS tape archive (permanent) 42 TB
08/2011 - 08/2012 Thumbnails ADS tape archive (permanent) 66 TB
08/2012 - 08/2013 Thumbnails ADS tape archive (permanent) 80 TB
08/2013 - 08/2014 Thumbnails ADS tape archive (permanent) 94 TB

Total storage requirements: 11 TB rolling cache, 47 TB / year permanent storage over 6 years (282 TB at the end of year 6)

  • UK Data Centre: light footprint:
    UK Data Centre: light footprint
  • UK Data Centre: heavy footprint:
    UK Data Centre: heavy footprint
  • UK Data Centre: heavy footprint with tape archive:
    UK Data Centre: heavy footprint with tape archive

-- ElizabethAuden - 23 Aug 2005

IAttachmentHistoryActionSizeDateWhoComment
GIFgifeSDO_DC_heavy.gif r1 manage 13.7 K 2005-09-29 - 17:51ElizabethAuden UK Data Centre: heavy footprint
GIFgifeSDO_DC_heavytape.gif r1 manage 14.6 K 2005-09-29 - 17:51ElizabethAuden UK Data Centre: heavy footprint with tape archive
GIFgifeSDO_DC_light.gif r1 manage 13.0 K 2005-09-29 - 17:51ElizabethAuden UK Data Centre: light footprint

View topic | View difference side by side | History: r7 < r6 < r5 < r4 | More topic actions
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback