eSDO 1211: Quicklook and Visualization Plan

This document can be viewed as a PDF.
Deliverable eSDO-1211
E. Auden, T. Toutain
23 August 2005

Introduction

The SDO mission will produce 2 TB of raw data every day. Although low and high level science products will only comprise a tenth of that volume, 200 GB per day represents a vast amount of information for a person to process. The eSDO quicklook and visualization plans have two goals: firstly, to help scientists search for and identify the SDO data they require, and secondly, to use this targeted search strategy to reduce the number of large data requests transferred across the network. Quicklook and visualization products will primarily take the form of thumbnail images, movies, and catalogues.

Visualization Tool

The JSOC team has identified a streaming tool that can be developed for visualization of AIA and HMI datasets. A user will open a browser-based GUI, choose either an HMI line-of-sight magnetogram or one of the 10 AIA channels, and select a start time and start cadence. The GUI will stream the AIA or HMI data product from the nearest local disc cache as a full solar disc, low resolution 512 by 512 image to the user's screen. The user will then be able to pan and zoom in space, time, and data product while studying evolving behaviour of solar features.

Zoom and Pan in Space

Zoom and Pan in Time

Switch Data Products

Quicklook Products

Images

As the science products are pulled from the US data centre to the UK, SDO processing software will extract images from all low level AIA and HMI science products archived in the UK data centre. Images will be converted to 200 by 200 pixel thumbnail GIFs of approximately 100 kB each and then archived for the duration of the SDO mission. Relevant metadata such as observation time, wavelength or magnetic data, and other science product information will be written to a searchable thumbnail catalogue. Thumbnail images will be produced for the following science products:

  • HMI line-of-sight magnetograms
  • HMI dopplergrams
  • HMI 20 minute averaged filtergrams (intensitygrams)
  • AIA low resolution, full disk images in 8 wavelengths
  • AIA high resolution, tracked active region images in 8 wavelengths (for ~1 - 10 active regions, depending on solar activity)

Movies

Movies shall be generated from thumbnail images on-the-fly by user requests. Users will specify the start time, end time and science products with which they wish to generate a movie; the relevant thumbnail images will be retrieved from the UK data centre and encoded as an MPEG file using the Berkeley MPEG encoder1, currently in use with AstroGrid solar tools. Movies will be cached along with start / stop time and science product metadata, but they will not be permanently archived.

Catalogues

Three catalogues will be generated to aid data searches: a thumbnail catalogue, a small event / coronal mass ejection (CME) dimming region catalogue, and a mode parameters catalogue. The thumbnail catalogue will contain information about each thumbnail image generated and archived in the UK data centre. Users will be able to search for thumbnail images based on observation time, science product type, wavelength (for AIA images) and NOAA active region number (for AIA tracked active images). The small event / CME dimming region catalogue will contain observation time, coordinate information, AIA science product name and other data for small solar events and CME dimming regions identified by the small event detection2 and CME dimming region recognition3 tools processing AIA data on a UK machine. Finally, the mode parameters catalogue will provide a table of frequency, lindewidth, amplitude and powers for a range of low l degrees.

User Access

User access to thumbnail images, generated movies and catalogues will all be available through the AstroGrid portal. The thumbnail and small event / CME dimming region catalogues will be searchable through an instance of the AstroGrid DSA4 software, while thumbnail images will be retrieved from the archive using an AstroGrid CEA5 tool and placed in the user's home computer or AstroGrid

MySpace
6 area.

In addition, a JSP page will be made available through the eSDO portal that will use the AstroGrid DSA and CEA webservices to display thumbnail images, movies, and catalogue results directly to a webpage. This page will provide simple forms that will allow users to first delineate start time, stop time, and science product type and then search the two quicklook catalogues, view thumbnails, or generate movies.

Technical Requirements

Software

Four types of software are required for the quicklook and visualization strategy described:

  1. Software to extract images from SDO science products and convert them into thumbnail GIFs
  2. The Berkeley MPEG encoder for movie generation
  3. A database such as MySQL to store tabular information for the thumbnail and small event / CME dimming region catalogues
  4. AstroGrid DSA and CEA modules

Hardware

The UK data centre will require a staging machine to pull data from the US data centre and load it into the UK archive (please see DataCentre1321 for more details). To avoid additional data transfers, this machine can also host the image extraction / thumbnail conversion software. A machine will also be required to host the MPEG encoder, database, and AstroGrid software. Because these three pieces of software will not require large data transfers, they can be hosted on either the staging machine or at a second location.

References

  1. Devadhar S., Krumbein C., Liu K., Smoot S. and Eugene H. The Berkeley MPEG Encoder. Berkeley Multimedia Research Company. http://bmrc.berkeley.edu/frame/research/mpeg/mpeg_encode.html. Last viewed 23/08/05.
  2. Please see PhaseASmallEventDetection for more information.
  3. Please see PhaseACMEDimmingRegionRecognition for more information.
  4. Astrogrid DSA, Data Set Access (formerly Publishers Astrogrid Library). http://www.astrogrid.org/maven/docs/1.0.1/pal/index.html. Last viewed 23/08/05.
  5. Astrogrid CEA, Common Execution Architecture. http://www.astrogrid.org/maven/docs/1.0.1/applications/index.html. Last viewed 23/08/05.
  6. Astrogrid MySpace (software component known as FileManager), http://www.astrogrid.org/maven/docs/1.0.1/filemanager/index.html. Last viewed 23/08/05.

-- ElizabethAuden - 23 Aug 2005

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | More topic actions...
Topic revision: r3 - 2005-09-26 - ElizabethAuden
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback