Tags:
create new tag
view all tags

Quarterly Reports

eSDO quarterly project reports from project manager:

2007

01 April 2007 - 30 June 2007

Staff: Andrew MDermott has joined the RAL effort to complete the Coronal Dimming algorithm.

Meetings and Conferences: Elizabeth presented the streaming tool at NAM 2007 in mid-April. Thomas Wiegelmann visited MSSL to work with Mike on the NLFFF mag extrap algorithm, and then Mike attended a mag extrap workshop in Paris in mid-June. Mike and Elizabeth also attended the June Heliophysics Knowledgebase workshop in Brussels and presented automated VOEvent generation results for the coronal loop recognition algorithm.

Technical Developments:

Algorithms:

All 9 algorithms are nearing completion. IDL wrappers have been written for the global (Birmingham) and local (Sheffield) helioseismology algorithms, and wrappers are underway for the MSSL algorithms. RAL's coronal dimming and small event algorithms are compiling and executing with fast time trials; final adjustments are underway to convert output formats from ASCII to FITS, place additional physical parameters in table extensions, and run the science cases listed on the algorithm deployment pages.

The magnetic extrapolation algorithm paralellized with MPI-CH2 is now running on National Grid Service parallel facilities. There is unlikely to be an AstroGrid CEA interface to the parallelized version of the code by end of September 2007, but users will have code plus documentation instructing them on 1) how to obtain a UK grid certificate and NGS account, 2) how to build the mag extrap code on NGS, and 3) how to execute the code in parallel on NGS.

Algorithm documentation will consist of 1) standardized comments inside algorithm code and 2) PDFs of finalized algorithm deployment wiki pages.

Visualization:

The streaming tool is now running as a server-side demo with EIT code. The June 30th deadline has not yet been met for saving movies to MySpace or viewing movies of instrument data other than EIT; this work is underway for completion in early August.

Data Centre:

An AstroGrid DSA module has been configured for the Postgres database holding sample data for the test installation of DRMS / SUMS at MSSL. DRMS / SUMS configuration will be completed over the summer, and then Elizabeth will work with developers at MPI Gottengin to set up DRMS / SUMS there. AstroGrid DSA and STAP modules will be hosted in the UK to connect to the MPI archive.

Project Issues: 3 months left of project; algorithm documentation / final code refactoring underway. Functionality of streaming tool must be finished over summer; post-launch data centre will not be hosted in UK, so AstroGrid integration must be done with MPI data archive.

Deliverables: By 30 September:

  • Algorithms: live AstroGrid CEA deployments of all 9 algorithms hosted at MSSL. Downloadable tarballs containing source code, documentation (PDFs of deployment wiki pages), test data, and IDL wrappers of algorithms placed on wiki.
  • Visuzalization tool: live multi-instrument streaming tool hosted at MSSL plus .war file of streaming tool code and documentation.
  • Data centre: documentation of local DRMS / SUMS installation, configuration, and integration with AstroGrid DSA and STAP. If possible, documentation of UK hosted DRMS / STAP connection to remote MPI DRMS / SUMS.

    01 January 2007 - 31 March 2007

    Staff: Additional effort has been added to the coronal dimming region recognition algorithm at RAL.

    Meetings and Conferences: An eSDO consortium meeting was held at RAL on 24 October 2006. In February, Sergei attended a Stanford workshop on local helioseismology algorithms in the JSOC pipelin. In spring and summer 2007, the eSDO project will be represented at NAM, an NLFFF workshop organized by JSOC and LMSAL, and 1st Heliophysics Knowledge Base workshop. In addition, Mike will participate in a coronal loops recognition "hare and hounds" exercise.

    Technical Developments:

    Algorithms:

    All algorithms have completed documentation for 3 - 5 science cases each on the wiki. 7 algorithms have completed both thorough unit testing and time trials for each of their 3 - 5 science cases; see AlgorithmDeployment pages and links to corresponding Deployment Results pages. Additional info:

    • Coronal loop recognition: code is included in JSOC coronal loops hare and hound exercise. Will test against LMSAL Heliophysics KB test data in June.
    • Magnetic extrapolation: code now available in single cpu / parallelized MPICH2. Time trials of MPI code will be conducted in late April 2007 ahead of collaboration with Thomas Wiegelmann's OpenMPI implementation of NLFFF. Mike will participate in NLFFF workshop.
    • Helicity calculation: bundled with mag extrap; results off test data by factor of 2
    • Small events detection: run time problems; hope to test against LMSAL Heliophysics KB test data in June.
    • coronal dimming region recognition: compilation issues. Hope to test against LMSAL Heliophysics KB test data in June.
    • Mode parameters: suitable for MDI data up to l=10; with improvements coming with SDO HMI data, hope to use up to l=50.
    • Local helioseismology inversion: multiple kernels now available; included in Feb. JSOC pipeline workshop.
    • Subsurface flow analysis: works with LHI and multiple kernels; included in Feb. JSOC pipeline workshop.
    • Perturbation map generation: works with LHI and multiple kernels; included in Feb. JSOC pipeline workshop.

    Visualization: The streaming tool has now been migrated to a serverside web based application after the initial clientside prototype exhibited slowness in data loading on wireless and home networks. Web based application offers following functionality:

    • query AstroGrid EIT STAP service for graphical data
    • display animation of selected graphical data in web page; user can
      • in movie mode
      • step forward and backwarks through frames
      • speed up or slow down animation speed
      • zoom in spatially, drag zoomed images around with google maps-style grabber (supercedes prototype spatial panning with clickable arrows)
      • and execute zoom / drag while movie is playing

    Updates by June 30:

    • increase number of STAP services queried - include EIT, CDS, MDI, hopefullly Hinode, maybe STEREO?
    • display selected images as JPEG gallery or save as MPEG
    • save JPEGs and MPEGs to MySpace
    • mission-specific SolarSoft processing?

    Data Centre: Future funding is uncertain for a UK-based SDO data archive, so the data centre prototype will focus on a European data centre. In late spring / early summer, eSDO will work with MPI Lindau to install JSOC DRMS / SUMS and Sloaney along with AstroGrid DSA and STAP. If test images are available, the STAP service will be added to the streaming tool.

    Project Issues: Uncertainty with future UK archive; fast approaching project end on 30 September 2007, final deliverables demonstrated in UK eSDO workshop plus LWS workshop in Boulder. Uncertainty for number of coronal feature / event recognition algorithms ready for testing against Heliophysics Knowledge Base by June.

    Deliverables: Completed: thorough unit tests (7 algs), completed science test time trials and comments (7 algs), completed 3 - 5 science case documentation (9 algs), server side streaming tool with movie mode functionality, spatial pan and zoom, and AstroGrid STAP integration.

    2006

    01 October 2006 - 31 December 2006

    no report

    01 July 2006 - 30 September 2006

    Staff: No change.

    Meetings and Conferences: Len and Elizabeth met with members of the UK coronal wave community on July 10 (minutes here). Len represented eSDO and the SDO PLS proposal at the UK solar community meeting in Manchester on 24 July. Although eSDO was not formally represented at SOHO-18, the meeting provided a chance to liase with JSOC colleagues from Stanford and Lockheed Martin. Mike Smith presented his coronal loop recognition algorithm at the Solar Image Processing Workshop on 6 September. Finally, Elizabeth gave a talk on eSDO data and algorithm VO integration at the UK e-Science All Hands conference on 21 September and again at the HELAS roadmap workshop on 27 September.

    An eSDO consortium meeting is planned for 24 October in the next quarter.

    Technical Developments:

    Algorithms: Developers have completed the first half of the requirements for the scientific testing scripts in the form of AlgorithmDeployment pages that explain algorithm functionality, compilation, execution, sample inputs, and sample outputs. At least one unit test has been added to each C class, and all functions will be thoroughly unit tested by the end of the next quarter. Further detail and test cases will be added to the science tests. The coming quarters will also focus on good documentation for algorithms. In addition. deliverable 2121, integration of algorithms with AstroGrid, is almost complete nearly 9 months ahead of schedule.

    Visualization:

    The streaming tool has been demonstrated at a number of meetings. Feedback and comments on future functionality are available at StreamingToolEvaluation. Note: at the end of this quarter, the movie maker has still not been completed. The code is currently being refactored from a java webstart application to a JSP page incorporating the image gallery tool so that users can choose to generate a thumbnail gallery, MPEG or animated GIF from the same web page.

    The delay in completion has occurred following tests with the streaming tool and image gallery applications demonstrating that downloading FITS files to a user's machine and processing them locally leads to slow display times. It is hoped that serverside processing that can also take advantage of the SUPERJANET network will show a noticable gain in speed for users. Collaboration with AstroGrid is currently underway to PLASTIC-enable the new image gallery / movie maker web page. Estimated completion date is 31 October 2006.

    Data Centre: The JSOC DMRS and SUMS systems have been refactored to work with Postgres as an alternative to Oracle. Postgres has been installed on msslxx, and loading the DRMS tables from Stanford to MSSL is underway. An evaluation of remote DRMS / SUMS operation between MSSL and Stanford will be ongoing until 22 December 2006. This evaluation will also explore integration with a possible European SDO data centre at Lindau following the HELAS workshop.

    Project Issues: Movie maker application currently behind schedule, but algorithm grid deployment ahead of schedule. SDO PLS SoI undergoing internal consideration.

    Deliverables: Complete - streaming tool evaluation, science test algorithm deployment pages. Incomplete - movie maker application. Next quarter: completed unit tests, detailed science tests, movie functionality in streaming tool.

    01 April 2006 - 30 June 2006

    Staff: No change.

    Meetings and Conferences: Elizabeth gave a project overview talk at UKSP / MIST in Aberystwyth on 11 April 2006. Len will present the SDO post launch support grant at the solar community meeting in London during the next quarter.

    Technical Developments:

    Algorithms: The first drafts of code for nearly all algorithms are now in CVS. The next quarter will introduct unit tests and continued refactoring to follow the eSDO C coding standard. In addition, developers and scientists will work together to develop scripts for scientific testing. These scripts will be written scripts for a user to follow, not automated batch scripts.

    Visualization: Java Webstart files are now publicly available for users to launch the StreamingTool, ImageGallery and ThumbnailMaker tools. The Streaming Tool and Image Gallery tools work in tandem with the AstroGrid Helioscope tool. Once a user has launched the Streaming Tool or Image Gallery tool, they can be registered with a PLASTIC hub such as the AstroGrid Client Runtime. Next, the user can search for data in Helioscope by entering start and stop times. Finally, by double clicking on the URLs returned by Helioscope, the user can automatically load the files as FITS images in the Streaming Tool or Image Gallery.

    The Streaming Tool prototype currently offers spatial pan and zoom facilities. The next iteration will allow users to view multiple FITS files in movie mode with play, rewind, step forward and step back facilities. eSDO scientists can add to a list of future enhancements, such as saving static images as jpegs or movie selections as MPEGs or IDL data cubes.A meeting of coronal seismology experts on 10 July 2006 (see CoronalSeismologyMinutes100706) yielded suggestions for quicklook wavelet and FFT analysis, using a mouse to drag a box over a region for further analysis, and viewing corresponding AIA and HMI images simultaneously (as overlays or galleries). An evaluation of future Streaming Tool development and enhancement will be completed in the next quarter.

    The Image Gallery prototype displays thumbnail images with corresponding filenames, instrument name and observation dates. This tool may evolve into a web page rather than a Java Webstart application so that image processing would be done on a remote server rather than on the client machine. In addition, this application would allow user searches through direct queries of a STAP (standard time access protocol) web service rather than through Helioscope. The next quarter will include development of a movie maker application that will likely follow the web page model rather than Java WebStart.

    Finally, the Thumbnail Maker tool can be run as a Java Webstart GUI or as a commandline application. This tool generates 256 by 256 pixel JPEGs from local FITS files or URLs and saves them to the user's machine. Each thumbnail includes a display of the original filename and observation date.

    Data Centre Network tests from the previous quarter have been completed between MSSL, UCL, RAL and Stanford using scp, rsync and sftp transfer protocols. A report of these tests can be found at NetworkLatencyTests. Future tests later in 2006 will test gridFTP between MSSL and Stanford.

    Project Issues: Expected code from outside partners was not available. Streaming Tool and Thumbnail Maker code were generated from the IVOA's FITS java package. Two algorithms await completed translation into C. Documentation and APIs for pipeline modules were discussed at the February SDO workshop in Monterrey, but no date is currently available for the availability of these items.

    Deliverables: Complete - first draft of code for all algorithms in CVS, Streaming Tool prototype, Image Gallery tool, network latency tests (late from previous quarter), thumbnail extraction tool. (late from previous quarter). Next quarter: scientific testing scripts, movie maker tool, streaming tool evaluation

    03 January 2006 - 31 March 2006

    Staff: No change.

    Meetings and Conferences: All eSDO developers plus three science advisors (Len Culhane, Mike Thompson and Andrzej Fludra) travelled to Monterey, California for the AIA / HMI joint meeting February 13 - 17. Elizabeth Auden gave an eSDO overview talk, Len Culhane spoke about the Solar-B mission (particularly the EIS instrument) as well as on coronal seismology, and Mike Thompson gave talks on local helioseismology. Andrzej Fludra spoke about automated detection algorithms for small events and CMEs.

    The developers liaised with several members of the solar community regarding algorithm code. Some of the eSDO algorithms will be put forward as test cases for the JSOC pipeline later this summer - likely candidates are the small event detection and coronal loop recognition codes. Elizabeth worked with Rick Bogart and Rasmus Larsen to arrange network tests between the US and UK and to make initial plans for deploying a UK instance of the JSOC data resource management system (DRMS). Elizabeth and Rasmus also discussed the initial prototype for the SDO streaming tool.

    SDO and eSDO overview talks were also given at the UK Solar Missions Forum at UCL on 23 February by Andrzej and Elizabeth. Another eSDO overview talk will be given in the next quarter at UKSP / MIST.

    Technical Developments: Developers are now able to upload code into CVS: see AccessToCVS and EclipseCVS. A risk management policy and plan have also been completed: see RiskPolicyAndPlan. The final list of eSDO algorithms was emailed to Deb Miller following the Monterey meeting as requested at the Phase A review. The Twiki unexpectedly required an upgrade for security reasons in late February; unfortunately, this work delayed the network tests and further development of the thumbnail labelling and extraction tool. However, both of these tasks are expected to finish a week after their 31 March deadline, so impact to the rest of the project will be minimal.

    Developments during this quarter will include first drafts of all algorithm code, initial deployment of the DRMS and a rapid prototype of the SDO streaming tool.

    Project Issues: Increased emphasis on coding and testing of algorithms, liaising with JSOC over DRMS and streaming tool.

    Deliverables: Network latency tests, thumbnail extraction tool. (Note: delayed as of 4 April 2006 - amend when finished.)

    2005

    01 October 2005 - 22 December 2005

    Staff: No change.

    Meetings and Conferences: The eSDO project team met for an October 6th consortium meeting in Sheffield (see ConsortiumMinutes061005), and the PPARC Phase A review was held on 28 November at the Medical Research Council in London. The panel was supportive of Phase A research and proposed Phase B activities with three recommendations:

    1. Define a risk analysis plan, particularly covering algorithms
    2. Increase effort on visualization tools, particularly evaluating datacubes for wavelet analysis
    3. Reduce emphasis on DEM algorithm
    ElizabethAuden has also joined the AstroGrid Deployment Advisory Group to integrate eSDO grid requirements with evolving AstroGrid installations.

    Technical Developments: Registry now uptodate with an external eXist database. New Sky Cat Server DSA modules deployed for testing database searches. Initial algorithm coding has commenced; CVS must be made available to external users as soon as possible. MSSL system administrators have advised that the eSDO server must upgrade to Fedora operating system in order to grant external CVS access; this will happen in early 2006. CUTest plans, risk analysis plan, and workpackage assignments must also be done in early 2006.

    Project Issues: Preparation for the Nov. 28 Phase A Review, preparation for February joint AIA / HMI meeting.

    Deliverables: No deliverables for this quarter.

    01 July 2005 - 30 September 2005

    The eSDO Phase A research year has finished, so project developers and scientists will now begin development of the algorithms, visualization tools and data centre implementations laid out in the PhaseAReport.

    Staff: No change.

    Meetings and Conferences: ElizabethAuden collaborated with SDO colleagues at Stanford and Lockheed Martin September 5 - 9 and then gave an overview talk, paper and poster at the 2005 UK e-Science All Hands conference in Nottingham on Sept. 19. Elizabeth and LenCulhane have met with ATLAS Datastore staff at RAL to include eSDO UK data centre designs with prototyping for Solar-B through August and September.

    Technical Developments: A new visualization tool has been proposed for eSDO development: the SDO streaming tool. eSDO developers with work with Stanford scientists to develop an image view that can pan and zoom on SDO images in both space and time. Also, all 11 proposed algorithms have technical writeups and initial codes.

    Project Issues: The last three weeks of this quarter concentrated on the final documentation for the Phase A deliverables - algorithm writeups, data centre implementation plan, and visualization tool descriptions. Once the documentation was finished, it was emailed to colleagues and interested parties in the UK solar community, UK grid community, and SDO mission in preparation for the Phase A Review.

    Deliverables: PhaseAReport and PhaseASummaryReport. The first report is a full document containing all eSDO deliverables plus appendices relating to eSDO prototypes using AstroGrid components. The summary report summarizes each full deliverable and provides links to the full report. Total deliverables for Phase A:

    • eSDO 1111: List of solar algorithms that eSDO institutions will develop and deploy as grid services. Completed 1 April 2005
    • eSDO 1121: Proposed solutions to each of the 11 algorithms detailed in eSDO 1111. Completed 30 September 2005.
    • eSDO 1131: Plan for integrating algorithms as grid services with AstroGrid, the JSOC pipeline, and SolarSoft.
    • eSDO 1211: Plans for quicklook products and visualization techniques including catalogues, thumbnail, gallery and movie generation, and the SDO streaming tool. Completed 30 September 2005
    • eSDO 1311: Design plans for implementing the UK data centre and integrating resource requests with the US data centre. Completed 1 April 2005
    • eSDO 1321: Design plans for integrating the UK data centre and, where possible, the US data centre with AstroGrid. Completed 30 September 2005

    01 April 2005 - 30 June 2005

    The eSDO project has now completed the third quarter of Phase A, and the final push in finishing the eSDO research deliverables will be undertaken in the upcoming quarter. Deborah Miller of PPARC will coordinate a community review of the eSDO project at the end of Phase A in late September or early October.

    Staff: In June Simon Martin handed over the RAL developer position to Vito Graffagnino. We wish Simon all the best as he moves into a new role at RAL. Vito will be working with Andrzej Fludra on the three image processing algorithms undertaken by the RAL team. The development team now stands as

    Meetings and Conferences: Lidia van Driel-Gesztelyi represented eSDO during the April 4-8 UKSP conference in Birmingham. In addition, the eSDO project team held a consortium meeting in Birmingham on May 4; minutes are available at ConsortiumMinutes040505. A meeting was with Karel Schrijver and Neal Hurlburt at UCL on June 22 to discuss AIA algorithms and data products, and those minutes can be viewed at AIAmeetingMinutes220605. Followup telecons will be held to discuss HMI data and algorithms in the fourth quarter.

    Over the coming months, eSDO will be represented by Elizabeth Auden at the July 11 AstroGrid consortium meeting, Len Culhane at the UCL Research Computing Forum on July 15, and by Elizabeth Auden again at the UK e-Sicence 2005 All Hands Meeting in a workshop on Data and Grid Computing on September 19.

    Technical Developments: Seven AstroGrid components have now been installed at MSSL. Installation and configuration notes for all modules can be found at AstrogridInstallation, and a portal tutorial is available at AstroGridTutorials.

    One of the topics of discussion at the May 4th consortium meeting was the overlap between quicklook product development and identification of products that should be stored in the UK data centre. These two issues are intertwined since the purpose behind quicklook (or "thumbnail" and "browse data") development is to provide low volume data that scientists can quickly view and determine which larger scientific datasets to download and process. Therefore, development of quicklooks will be concentrated on products that will actually be queriable or viewable through US or UK data centres. Many ideas were discussed for UK data centre storage, including

    • A sampled archive of AIA and HMI filtergrams over the lifetime of the mission
    • Movies of AIA and HMI filtergrams (or higher level images)
    • Thumbnail images of high level products such as maps and magnetograms

    In order to ascertain which quicklook products and visualization capabilities the UK community would find most useful to aid efficient searching of the full SDO archive, a questionnaire has been submitted to the July issue of UK Solar News. The answers from the following questions will speed identification of which quicklook products should be generated and stored. In addition, size and cadence estimates will shape the amount (and cost) of storage required for the UK data centre.

    1. Do you currently use data from TRACE, EIT, or MDI?
    2. f so, do you usually request low level data products (such as TRACE hourly files or MDI magnetograms) or data products that have undergone significant reduction (such as TRACE calibrated JPEG images or MDI smoothed velocity reports)?
    3. What would help you search the SDO archive more efficiently?
    • thumbnail images of low level data (magnetograms, filtergrams)
    • thumbnail images of high level data (calibrated images)
    • movies of a month's worth of low level data sampled once a day
    • movies of a month's worth of low level data sampled once an hour
    • feature and event catalogues
    • other data product

    Len Culhane, Elizabeth Auden and MSSL system administrator Paul Lamb have set up a dialogue with David Corney and colleagues at the ATLAS Petabyte storage facility at RAL to discuss the needs of the UK data centre. This facility will be able to provide both online disc caches as well as grid-accessible permanent tape storage. David has provided initial cost estimates for two models of eSDO data storage, and more details can be viewed at DataCentreCosts. Elizabeth, Len and Paul will liase further with David following UK Solar News community input to the type and cadence of AIA and HMI data products that could usefully be stored in the UK data centre. It is expected that firmer estimates of eSDO storage requirements will be reached in mid-August, so more detailed financial models for ATLAS storage will be made then. Costs and data storage models for the UK data centre will be published in eSDO deliverable 1321.

    Project Issues: The issue of algorithm language choice has now been resolved - the algorithms will be coded in C. Three methods of distribution have been identified for the algorithms: inclusion in HMI and AIA pipelines, C modules wrapped in IDL and distributed through the MSSL SolarSoft gateway, and, if funding is available, UK hosting of C modules wrapped as AstroGrid CEA tools. As algorithms in the SDO pipeline must run with a predefined set of input conditions, distribution through SolarSoft and AstroGrid CEA will allow users more control over raw data processing with variable input parameters.

    The issue of UK data centre storage has also been addressed through the ATLAS Petabyte facility at RAL; this brings up the new issue of what to store in the UK data centre now that approximate storage costs are known for different data storage models. Current estimates allow for 1) an online disc cache partitioned to store the most recent SDO data as well as the most popular SDO datasets (to remove traffic pressure from the SDO data centre in California), and 2) grid-accessible tape storage to archive useful quicklook products and data sampled over the lifetime of the SDO mission. It is now known that all AIA low and high level products will be accessible through the US data centres (at Stanford and Lockheed Martin respectively). All HMI low level products and some HMI high level products will be accessible at Stanford - communication with the HMI team is now determining which high level products would not be archived at Stanford and whether it would be useful to store these products in the UK.

    A final project issue revealed in the third research quarter deals with pursuit of multiple algorithm solutions. Certain eSDO algorithms, particularly DEM computation, coronal loop recognition and magnetic field extrapolation have several possible solutions. At the end of Phase A eSDO teams will have select one or two solutions to pursue in earnest. A joint AIA / HMI meeting will take place during February in California, and breakout sessions have been suggested to discuss specific algorithms. eSDO developers should attend these breakout sessions where possible.

    Deliverables: Templates for all proposed algorithm solution deliverables have been created for eSDO 1121; a full example can be viewed as PhaseALoopRecognition. No finished deliverables were due for the third research quarter. The fourth quarter will see completion of the remaining Phase A deliverables: proposed algorithm solutions, data centre and algorithm grid integration plans, and summary of quicklook products to be generated.

    01 January 2005 - 31 March 2005

    The eSDO project has completed the second research quarter of Phase A. All developer positions have been filled, a prototype system is underway, and plans for solar algorithms, a US data centre and a UK data centre are solidifying.

    Staff: After interviews at the University of Birmingham in February and the University of Sheffield in March, all developer positions have now been appointed:

    Bill Chaplin at Birmingham will also be working with the science advisory team.

    Meetings and Conferences: The eSDO project was represented at two meetings during the first quarter of 2005. ElizabethAuden presented an overview of the project at the UK Solar Missions Forum at Ral on February 8, and she also attended the SDO JSOC peer review at Stanford University on March 17. LidiavanDriel-Gesztelyi will speak about the SDO mission, including UK involvement through hardware and eSDO software, at the UKSP meeting in conjunction with NAM next week, April 4-8.

    Technical Developments: MikeSmith and ElizabethAuden are building a prototype eSDO system from Astrogrid components. This software is installed on msslxx at MSSL, and it will provide a testbed for eSDO / Astrogrid integration through a number of modules: web portal, queriable data centre, registry, mySpace storage area, duplicates of the Astrogrid solar movie maker and solar overlay tools, workflow manager, and eSDO community. Accounts will be created for all eSDO project members. Accounts may also be requested for AIA and HMI team members and colleagues of the science advisory team.

    Literature searches for 13 solar algorithms began in January, and the eSDO wiki currently holds references, sample code, and some sample data for 6 algorithms: loop recognition, magnetic field extrapolation, helicity computation, small event detection, DEM computation, and CME dimming region recognition. Final reports on all algorithms will be completed by the end of September. Following the SDO JSOC peer review in March, it has emeged that the AIA and HMI instrument teams would be interested in including all eSDO algorithms (with the exception of the helioseismology workbench) in the JSOC pipeline.

    Finally, initial implementation plans for data centres in the US (at Stanford University) and the UK (RAL or MSSL) have been completed. These plans discuss which SDO data products will be retrievable through the eSDO data centres in both countries, as well as length of storage time. eSDO will work with Rick Bogart and Karen Tian at Stanford to integrate the SDO database and filesystem with an instance of the Astrogrid PAL (Publisher's Astrogrid Library).

    Project Issues: The two project issues mentioned in the last quarterly report, algorithm language choice and location of the UK data centre, are being resolved. If the eSDO algorithms are included in the SDO JSOC pipeline, the logical language choice would be C as other pipeline modules will be written in C. LenCulhane and ElizabethAuden will meet with the RAL petabyte storage facility team in late April to discuss use of the RAL storage facility for the UK data centre. Representatives from the MSSL computing group will also be present, so the discussion will also explore whether the data storage system that will be built at MSSL for Solar-B would be a viable candidate for the eSDO data centre.

    Deliverables: Two deliverables have been completed as formal documents. DataCentreImplementation1311 discusses data centre implementation plans for US and UK eSDO data centres. Further technical details will be explored in deliverable 1321. SolarAlgorithms1111 describes the 13 algorithms that will be implemented by eSDO developers. A full report of each algorithm will be available in deliverable 1121; plans to integrate virtual observatory access to these algorithms will be detailed in deliverable 1311.

    2004

    01 October 2004 - 31 December 2004

    The budget for the eSDO project officially began on 01 October 2004 as PPARC PP/C502930/1. Four UK institutions will contribute to the project: MSSL-UCL, RAL, the University of Birmingham and the University of Sheffield. The three year project will encompass a one year research phase (Phase A) and a two year implementation phase (Phase B). The goals of the eSDO project are to deploy solar algorithms as grid applications, design quicklook solar data products that will enable high speed grid searches, and to integrate SDO data with the grid.

    Staff: ElizabethAuden of MSSL is the UK project manager. The science advisory team consists of LenCulhane (MSSL), YvonneElsworth (Birmingham), AndrzejFludra (RAL), RichardHarrison (RAL), MichaelThompson (Sheffield), and LidiavanDriel-Gesztelyi. During the first quarter, SimonMartin joined as a developer from RAL, and MikeSmith accepted a developer role at MSSL to begin in 2005. Two further developers will be hired at Birmingham and Sheffield for a start date of 01 April 2005.

    Meetings and Conferences: The eSDO project has been publicized at four meetings in 2004. A poster was presented as the LWS Workshop in March, at UKSP in April, at the UK e-Science All Hands meeting in September, and at the NASA VO Workshop in October during the first eSDO quarter. After a kickoff meeting in May, the science advisory team, ElizabethAuden and SimonMartin met at UCL to discuss developments for solar algorithms, the data centre, and grid integration. The science advisory team has advised that integration with EGSO in addition to Astrogrid would be beneficial to the UK solar community. In addition, the project should endeavor to provide talks and / or papers at conferences in addition to posters. Minutes of the UCL meeting can be read at ConsortiumMeetings231104.

    Technical Developments: A linux machine has been purchased and installed at MSSL. This machine - msslxx - will be used for demonstrations, CVS, algorithm development, and data centre testing. Msslxx runs red hat linux, and installed software includes Maven, Tomcat 5.0.28, Java JDK 1.4.1, and IDL 6.0. SolarSoft and the Astrogrid PAL will be installed in quarter two. In addition, two Sony VAIO laptops have been purchased for ElizabethAuden and MikeSmith. Laptops will be purchased for developers at Birmingham and Sheffield at the end of quarter two or the beginning of quarter three.

    Project Issues: There are currently two outstanding issues under investigation for eSDO: choice of language for algorithms and location of the Astrogrid PAL with respect to the SDO data storage areas at Stanford and / or Lockheed Martin. Current candidates for algorithm language include IDL, java, C, and FORTRAN. Each of these languages can interface with a web service; additional investigation is required to ascertain which of the proposed solar algorithms will benefit from a GUI instead of a commandline interface. As for the location of the PAL, ideally this software can be installed and maintained at Stanford. However, other possible locations include MSSL and RAL.

    Deliverables: One project deliverable has been achieved three months early during quarter one: a list of proposed SolarAlgorithms to write and deploy.

    -- ElizabethAuden - 04 Apr 2006

  • Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | More topic actions
    Topic revision: r15 - 2007-07-05 - ElizabethAuden
     
    This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
    Ideas, requests, problems regarding TWiki? Send feedback