create new tag
view all tags

Quicklook Products: Early Questions

Points for discussion:


1 HMI AIA JSOC Ground Data System Plan Overview (HMI-S019 / 10 Nov 2004) http://hmi.stanford.edu/doc/SOC_GDS_Plan/JSOC_GDS_Plan_Overview.pdf

2 HMI Data products (HMI-S022 / 16 Nov 2004) http://hmi.stanford.edu/doc/SOC_GDS_Plan/HMI_pipeline_JSOC_dataproducts.pdf

3 Atmospheric Imaging Assembly for the Solar Dynamics Observatory - Concept Study Report / 19 Feb 2004 AIA00436_AIA_Science_Plan.pdf

1. Should we make quicklook products for all the high level HMI and AIA data products?


There are 12 high-level (Level-3) HMI data products specified in section 2.5 of Ref. 1 (see also Figure 4). Ref. 2 shows that the specifics of these are still to be defined, and so a best guess has been made at the likely output format of each and its suitability as a quicklook product. Have e-mailed Rick Bogart at Stanford for clarification.

*Data Product                                    Likely Output                       Suitable for quicklook (Y/N)*
Internal Rotation                          Image - function of latitude and radius             Y
Internal Sound Speed                       Image - function of latitude and radius             Y
Full-disk velocity and sound speed maps    Image                                               Y
Carrington synoptics                       Image                                               Y
High-res velocity and sound speed maps     Image                                               Y 
Deep-focus velocity and sound speed maps   Image                                               Y
Far-side activity index                    F10.7 or SSN index                                  N
Line-of-sight magnetic field maps          Image                                               Y
Vector magnetic field maps                 Image                                               Y
Coronal magnetic field extrapolations      Superimposed image                                  N
Coronal and solar wind models              Movie                                               Y
Brightness images                          Image                                               Y 

The AIA data pipeline (see Figure A15 of Ref.3) shows that the following high-level data are to be made available as quicklook products:

Irradiance curves
Low-resolution summary movies  (1024 x 1024 pixels)
Full resolution extract event/feature movies
Full resolution AR extract movies
Reconstructed Temperature Maps
Coronal field-line models

Note1 : need to determine final list of JSOC pipeline level 1-3 data products. Current thinking is that JSOC will construct / host level 1 - 3 quicklook products in Stanford data centre.

Note 2: some of the above products (ie, Carrington synoptics) are quicklooks in their own right - rather than thumbnailing an image, any further work on "quicklooking" these products may need a new way of arranging basic data. - ECA, 08/02/05

2. If so, should they be stored in a data centre in the US or in the UK?

Depends upon where the HMI Level-3 data products themselves are archived, if at all. Para 8, Section 2.5 of Ref.1 indicates that a firm decision has yet to be made as to whether they will be stored or created on demand. If the latter, then the quicklook products become redundant.

Note: If data products are created on demand but only stored online for 3 months, it may sensible to generate a quicklook anyway, store it for longer than the data product (store permanently?), and indicate whether the data product exists online currently or was once created but no longer exists online. - ECA, 08/02/05

3. What about quicklook products for data products that come out of the eSDO algorithms - should every algorithm produce a quicklook , or will only some of the algorithms lend themselves to sensible quicklooks?(Example: the output images from the loop recognition and magnetic field extrapolation could have quicklooks made from them, but as the helicity computation output is an integer, there's not much quicker you could get! Unless we were to build up a full solar disk map of helicity numbers.)

I think we need to establish where each of the eSDO algorithms is to be deployed in the HMI/AIA data processing pipelines. Some clearly form part of the processes that produce the Level-3 HMI data and AIA quicklook products. For example, Figure 4, of Ref.1 shows Coronal Magnetic Field Extrapolations as one of its data products and is presumably created from the MSSL algorithm of the same name. While Figure 5, Ref.1 shows MSSL's Loop Recognition and Field Line Extrapolation algorithms as part of the process for producing AIA quicklook products. What of the other algorithms ? Are these used to post-process HMI Level-3 data products and so produce data products of their own, or are they deployed at a lower level ?

Note: eSDO algorithms will be separate services hosted in the UK. They will not be part of the JSOC pipeline.

4. Should we store quicklooks for every post-algorithm data product at a UK data centre, regardless of whether the data product itself is stored? Should storage have a finite (ie, 3 months) or permanent (ie, till we run out of funding) lifetime?

Regarding the first question, would the UK data centre be the sole repository for the quicklooks produced or a mirror site ? As for finite v permanent storage, I guess that depends upon predicted use v future storage capacity.

The UK data centre would store only eSDO post-algorithm quicklooks, not JSOC level 1-3 quicklooks, so it would be the sole repository rather than a mirror. Probable storage locations include MSSL, UCL Gower St. campus, or RAL. - ECA, 08/02/05

-- ElizabethAuden - 06 Jun 2005

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | More topic actions
Topic revision: r1 - 2005-06-06 - ElizabethAuden
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback