Tags:
create new tag
view all tags

Weekly Reports - MikeSmith

November 2006

Week Ending - 17/11/06

Re-visited Helicity code. Double-checked initialisation.c array indexing v FORTRAN code original.

Objectives for next week

eSDO:

Complete array index checking. Request test cases for Helicity and Magnetic Extrapolation from RIck DeVore and Thomas Wiegelmann respectively.

Hinode (Solar-B):

Configure EIS project machine if available.

Contact Jonathan Cirtain (XRT) and Tim Folkes (RAL).


Week Ending - 10/11/06

Objectives for next week


Week Ending - 03/11/06

Objectives for next week


October 2006

Week Ending - 27/10/06

Attended eSDO consortium meeting at RAL. Gave algorithm progress report.

Continued with Hinode (Solar-B) website - database integration.

Objectives for next week


Week Ending - 20/10/06

eSDO:

Experimented with gnuplot as a visualization tool for NLFFF output data. Possible replacement for IDL 'read_relax.pro' code.

Hinode (Solar-B):

Added JDBC. Can now perform SQL queries through the EIS website.

Objectives for next week

eSDO:

Continue gnuplot investigations.

Hinode (Solar-B):

Continue website development. Convert JSP java code to JAVA servlet.


Week Ending - 13/10/06

Hinode (Solar-B):

Tested populating MySQL database with EIT data files.

Objectives for next week

Hinode (Solar-B):

Add JDBC to allow MySQL access through website JSPs.


Week Ending - 06/10/06

Hinode (Solar-B):

Continued with Solar-B website construction. Updated MySQL table structure to reflect latest EIS FITS file header keywords.

Objectives for next week

eSDO:

Request test data/science cases from Rick DeVore to verify Helicity algorithm.

Make hard-coded input/output file references commandline arguments for Magnetic Extrapolation (ME) and Helicity codes. Correct ME file read error.

Hinode (Solar-B):

Create table for XRT FITS file header keywords. Also, create secondary tables to hold keywords & metadata temporarily while EIS products are not under ADS archive. Link tables as appropriate and look at URL creation.

Continue with Solar-B website. Install on new EIS pc if available.

Contact Tim Folkes at RAL, and Markus Aschwanden at Lockheed.


September 2006

Week Ending - 29/09/06

Completed first draft of eSDO MSSL algorithm unit tests. Updated software & test files added to CVS.

Objectives for next week

Continue with Solar-B website construction.


Week Ending - 22/09/06

Started work on eSDO MSSL algorithm unit tests.

Objectives for next week

Complete algorithm unit test suite.


Week Ending -15/09/06

Checked array indexing in Helicity 'C' code versus FORTRAN 90 code.

Continued work on EIS Solar-B website.

Objectives for next week

Begin work on MSSL algorithm unit tests.


Week Ending - 08/09/06

Attended Solar Image Processing III workshop at Trinity College, Dublin.

Objectives for next week

Review array indexing in Helicity 'C' code.


Week Ending - 01/09/06

Completed eSDO MSSL algorithm deployment pages.

Objectives for next week

SIP III workshop - Dublin.


August 2006

Week Ending - 25/08/06

Continued with Solar-B EIS website development.

Objectives for next week

Modify Magnetic Extrapolation and Helicity algorithms to read FITS file input and write FITS file output.

Update eSDO MSSL algorithm deployment notes on Twiki.

Investigate 'cfitsio' FITS file copy problem noted by Marc DeRosa.

Continue with Solar-B EIS website.


Week Ending - 18/08/06

Added Footpoint table to Loop Recognition FITS output file and added to CVS.

Released latest version of Loop Recognition algorithm code to Stanford (Marc DeRosa & Rick Bogart).

Continued with Solar-B EIS website development.

Attended Solar-B EIS meetings.

Objectives for next week

Continue with Solar-B EIS website.


Week Ending - 11/08/06

Continued with Solar-B website development.

Objectives for next week

Continue with Solar-B EIS website.


Week Ending - 04/08/06

Continued with Solar-B website development.

Objectives for next week

Continue with Solar-B EIS website.


July 2006

Week Ending - 28/07/06

Continued with Solar-B website development.

Objectives for next week

Continue with Solar-B EIS website.


Week Ending - 21/07/06

Discussed ADS-MSSL interface in detail.

Completed first draft of Perl script to read FITS headers and communicate with MySQL database on msslxv.

Started Solar-B website design.

Objectives for next week

Begin coding website. Decide upon hand-coded HTML approach or use of proprietary web design IDE.

Ensure configuration of Solar-B DSA on msslxv is global, not local.

Re-populate MySQL database on msslxx with eSDO data. Check it can be accessed through AG DSA running on msslxx.

Re-visit Helicity code.


Week Ending - 14/07/06

Attended EIS team meeting.

Installed new AG DSA on server msslxv to support SOLAR-B/EIS website.

Objectives for next week

Prepare script to read Solar-B FITS file metadata and write to MySQL database, and generate URLs.

Begin Solar-B/EIS website design.


Week Ending - 07/07/06

On holiday.

Objectives for next week

Install new AG DSA on msslxv to support Solar-B/EIS activities.


June 2006

Week Ending - 30/06/06

Further experimentation with the Helicity code. Added first draft to CVS.

Attended Solar-B meeting. Discussed ADS-MSSL interface.

Objectives for next week

On holiday.


Week Ending - 23/06/06

Added first draft of Loop Recognition and Magnetic Extrapolation code (for single CPU) to eSDO CVS.

Continued refining Helicity code.

Objectives for next week

Continue with Helicity code refinement.

Continue conversion of OPENMP to MPI in Optimization code.

Attend Solar-B MSSL-ADS interface meeting.


Week Ending - 16/06/06

Completed initial Helicity conversion. Started checking/correcting array indexing.

Experimented with eSDO CVS.

Attended Solar-B meeting to discuss scope of web services for the project.

Objectives for next week

Add tested eSDO algorithm code to CVS.

Continue with Helicity algorithm. Contact Rick DeVore for test data.

Investigate new AstroGrid tools, eg 'Plastic', 'Helioscope' and 'Aladin', and check their suitability for Solar-B web services.


Week Ending - 09/06/06

Continued Helicity code conversion/compilation.

Objectives for next week

Complete Helicity conversion and check array indexing is correct. Attempt to compile and run across two processors using existing Optimization test data as input.

Re-visit Preprocessor code.

Contact Rick DeVore for specific details about some of the variables used in the initialization and helicity diagnostics routines of his FCTMHD3D software.

Arrange informal Solar-B meeting to discuss web services for the project.


Week Ending - 02/06/06

Reviewed latest Optimization software. Revisited Azimuthal Disambiguation conversion.

Attended Solar-B team meeting.

Objectives for next week

Continue with Helicity code conversion.


May 2006

Week Ending - 26/05/06

Started preparing test files (FITS format) to check the efficiency of the Loop Recognition algorithm.

Received updated extrapolation code from Thomas Wiegelmann + test data for the Preprocessing step. Latest extrapolation code uses MultiGrid approach.

Objectives for next week

Look at Wiegelmann updates in detail.


Week Ending - 19/05/06

Managed to get 'Optimization' code running across two processors (msslxx + laptop) under MPI. Starting adding MPI constructs to code.

Investigated public key access to CVS on msslxx with Paul Lamb.

Objectives for next week

Continue with MPI conversion/experiments.


Week Ending - 12/05/06

Continued with Helicity code conversion. More information about the workings of FCTMHD3D requested from Rick DeVore.

Modified backup script to include 'griduser' scripts in daily save.

Objectives for next week

Experiment with MPI on the network and start converting 'Optimization' code from OPENMP to MPI.


Week Ending - 05/05/06

Started extracting Helicity code from Rick DeVore's FCTMHD3D FORTRAN software and converting to 'C'.

Requested further information from Rick DeVore about the structure/workings of the Helicity code.

Objectives for next week

Tie-up Helicity code with Rick DeVore's research papers.

Re-visit Ambiguity Resolution code.

Convert OPENMP commands to MPI in Optimization code.


April 2006

Week Ending - 28/04/06

Continued NLFF Preprocessing code conversion.

Objectives for next week

Start looking at Helicity code in more detail.


Week Ending - 21/04/06

Started converting NLFF Preprocessing code from IDL to 'C'. Additional Laplace operator code required.

Objectives for next week

Complete conversion of NLFF Preprocessing code. Compile, create test data and integrate.


Week Ending - 14/04/06

Further Ambiguity Code conversion work.

Objectives for next week

Integrate Ambiguity Code with FFTW (Fast Fourier Transform) library.

Start conversion of NLFF Preprocessing code.


Week Ending - 07/04/06

Continued with Ambiguity Code conversion.

Received copy of NLFF Preprocessing code from Thomas Wiegelmann.

Objectives for next week

Complete Ambiguity Code conversion and begin testing.

Analyse NLFF Preprocessing code.


Week Ending - 31/04/06

Continued with Ambiguity Code conversion.

Contacted Thomas Wiegelmann to request NLFF Preprocessing code.

Obtained e-science digital certificate from UCL.

Objectives for next week

Finish Ambiguity Code conversion and begin testing.


March 2006

Week Ending - 24/03/06

Continued with Ambiguity Code conversion.

Objectives for next week

Obtain e-science digital certificate.

Contact Thomas Wiegelmann to discuss Green Solver function in the 'Optimization' code and to request a copy of his NLFF Preprocessing code.


Week Ending - 17/03/06

Continued conversion of Georgolis' IDL Ambiguity code to 'C'.

Objectives for next week

Continue Ambiguity code conversion.

Tidy-up Loop Recognition code and add to eSDO CVS.

Obtain e-science digital certificate from UCL.

Contact Thomas Wiegelmann about pre-processing vector magnetogram data and benefits of pre-computing square roots in 'green' function.


Week Ending - 10/03/06

Installed and experimented with MPICH2 open-source parallel computing software.

Continued conversion of Georgolis' IDL Ambiguity code to 'C'.

Objectives for next week

Continue with Ambiguity code conversion.

Analyse Rick DeVore's Helicity code in detail.


Week Ending - 03/03/06

Looked at the benefits of precomputing square roots in the 'green' function in bfield.c. Three-fold speed-up for pixel data cubes up to 40 pixels, thereafter file size and memory become a problem.

Received a copy of Rick DeVore's Helicity code (based on NLFF computations).

Objectives for next week

Analyse Rick DeVore's code and see how this can be incorporated into the MSSL Helicity algorithm.

Set up MPI on local machine(s) and start adapting the Optimization code to run as a distributed process. Can the precomputed square roots data be split across several machines ?


February 2006

Week Ending - 24/02/06

Profiled Wiegelmann's NLFF Optimization code. Over 90% of processing time is spent in the 'green' function in the bfield.c module.

Obtained copy of Tom Metcalf's disambiguation code.

Objectives for next week

Investigate ways of reducing the processing required in the 'green' function.


Week Ending - 17/02/06

Attended AIA/HMI Team Meeting in Monterey, California (13/02/06 - 17/02/06).

Objectives for next week

Add Loop Recognition code to CVS and contact Paul Lamb about read permissions to all branches of the CVS tree.

Write-up notes from the team meeting.

Review Jim McTiernan's NLFF presentation and re-examine Grad Rubin NLLF extrapolation method.

Continue with NLFF profiling and parallelization.

Contact Thomas Wiegelmann about Optimization pre-processing technique (timings etc).


Week Ending - 10/02/06

Experimented with several profiling tools, including 'Google-Perftools', which proved the easiest to install and use ( admin rights not required). The profiler provided useful info. about the Loop Recognition algorithm and should give an insight into where the NLFF Magnetic Extrapolation bottlenecks lie.

Experimented with several parallelization protocols, including Open-MPI and PVM. PVM proved the easiest to install - several simple example programs were run successfully across the msslxx and msslai servers (under my username).

Objectives for next week

Attend AIA/HMI team meeting in Monterey.


Week Ending - 03/02/06

Prepared for and attended eSDO consortium meeting at UCL.

Experimented with Open-MPI installation.

Objectives for next week

Continue with investigation into parallelization techniques.


January 2006

Week Ending - 27/01/06

Started re-writing AD code in 'C'. Can now read data stored in IVM Hawaii Raw format into 'C' structures.

Completed detailed write-up of MSSL eSDO algorithms.

Objectives for next week

Continue with AD coding and investigate divide-by-zero error within IDL code.

Continue profiling Wiegelmann's 'optimization' code and look at possible installation of Open-MPI on MSSL network.

Prepare for next eSDO consortium meeting at UCL (3rd Feb '06).


Week Ending - 20/01/06

Informal chat with Len & Lidia about MSSL algorithm progress. Lidia to try and obtain Helicity code based on NLFF model.

Started experimenting with Manolis Georgoulis's 'Azimuthal Disambiguation' (AD) IDL code - copy received this week.

Tidied-up eSDO project server (msslxx) following OS upgrade to Fedora Core 2.

Objectives for next week

Re-write AD code in 'C'.

Complete MSSL algorithm status write-up.

Chase MySQL database upgrade on msslxx.

Continue Loop Recognition/Magnetic Extrapolation algorithm work in background.


Week Ending - 13/01/06

Researched similarities/differences between parallel computing protocols (OpenMP, MPI etc).

Began profiling Wiegelmann's 'Optimization' code in search of potential bottlenecks.

Objectives for next week

Investigate ways to improve Loop Recognition linkage process.

Start researching MPI command set and continue profiling 'Optimization' code.

Google for/request 'Azimuthal Disambiguation' code.


Week Ending - 06/01/06

Tracked down software over-write problem in Loop Recognition algorithm. Modified code to cater for all FITS pixel resolutions (Char, Short, Long, Float and Double) and input FITS images of differing dimensions.

Objectives for next week

Modify Loop Recognition code to allow for inverse images, i.e. dark loops on bright background. Add table of Loop Footpoints to output FITS file.

Modify Magnetic Extrapolation algorithm to overlay extrapolation data onto original magnetogram image.


December 2005

Week Ending - 16/12/05

Installed new external eXist server on msslxx. Investigated 'out of memory' problem. Not resolved but temporary workaround adopted (restricted the number of results returned from a query).

Integrated 'cfitsio' library with Loop Recognition algorithm to allow results to be written in FITS file format.

Objectives for next week

Refine Loop Recognition algorithm.

Start looking at Magnetic Extrapolation algorithm in detail (Wiegelmann pre-processing, Azimuthal Disambiguation & code optimization).


Week Ending - 09/12/05

Researched 'cfitsio' library functions as a replacement for the current 'XFITSview' interface for the Loop Recognition algorithm.

Objectives for next week

Integrate 'cfitsio' library with existing Loop Recognition code.


Week Ending - 02/12/05

Continued testing Loop Recognition algorithm with improved curve detection additions.

Objectives for next week

Link Loop Recognition code with cfitsio library (replace current X-window FITS display method).

Investigate potential bottlenecks in NLFF Magnetic Extrapolation code and experiment with Fortran NAG libraries on msslxx (post OS-upgrade).


November 2005

Week Ending - 25/11/05

Backed-up msslxx 'griduser' directory/sub-directories on DVD and under /disk/d4/griduser on msslxx.

Started experimenting with GNU profiler 'gprof' and looking at the NAG 'C' library.

Continued with Loop Recognition algorithm code.

Objectives for next week

Further experimentation with Loop Recognition algorithm.

More detailed look at NAG libraries and code profilers.

Re-install 'griduser' directory (assuming Fedora OS upgrade goes ahead).


Week Ending - 18/11/05

Documented eSDO server backup process on eSDO tiwki.

Added DVD diskspace check to backup script.

Perl script added to $CATALINA/logs to rotate 'catalina.out' file.

Objectives for next week

Continue with Loop Recognition algorithm.

Investigate NAG libraries for Magnetic Extrapolation algorithm.

Install new eXist database "fix".

Investigate loss of SSH during nightly eSDO server backup.


Week Ending - 11/11/05

Completed & tested eSDO server backup script.

Objectives for next week

Add DVD diskspace check to backup script.

Document backup process on eSDO twiki.

Continue with Loop Recognition algorithm and experiment with script to create daily log file from TOMCAT catalina.out file.


Week Ending - 04/11/05

Installed DVD read/writer on 'msslli' and experimented with DVD burning from the commandline.

Starting writing BASH script for eSDO server backups.

Objectives for next week

Finalise backup script.

Return to Loop Recognition algorithm.

October 2005

Week Ending - 28/10/05

Further experimentation with 'Improved Curve Tracing' modifications.

Objectives for next week

Install DVD read/writer on 'msslli' and experiment with eSDO server ('msslxx') backups.


Week Ending - 21/10/05

Started adding and experimenting with 'Improved Curve Tracing' modifications (angle change limits + Radon transform).

Objectives for next week

Continue experimenting with curve tracing improvements.

Experiment with STILTS library.

Investigate MySQL clustering.

Install DVD read/writer on 'msslli' and experiment with eSDO server ('msslxx') backups.


Week Ending - 14/10/05

Investigated different means of formatting ASCII files returned by AG workflows.

Continued coding Loop Recognition algorithm.

Objectives for next week

Continue with algorithm coding and testing.

Experiment with STILTS library and create CEA app to format ASCII workflow results.

Week Ending - 07/10/05

Attended eSDO consortium meeting in Sheffield.

Completed Powerpoint presentation for consortium meeting.

Added fixes for AG Release 1.1 problems (see last week).

Objectives for next week

Continue algorithm coding.

Look into formatting plain text files returned by an AG workflow.

Learn about CuTest.


September 2005

Week Ending - 30/09/05

Completed AstroGrid Release 1.1 upgrade on eSDO server (msslxx).

Completed documentation for the AG Release 1.1 upgrade - 'AstroGridInstallationV11' on the Twiki.

Objectives for next week

Investigate outstanding AG Release 1.1 problems:

  1. Non-harvesting Registry.
  2. Error message displayed when 'Properties' button clicked.

Prepare presentation for eSDO consortium meeting at Sheffield.


Week Ending - 23/09/05

More MATLAB -> 'C' conversion.

Start upgrading eSDO server (msslxx) AstroGrid components to Release 1.1.

Objectives for next week

Complete AstroGrid upgrade and document.

Continue (complete ?) MATLAB conversion.


Week Ending - 16/09/05

Continued with MATLAB -> 'C' conversion.

Objectives for next week

Continue with MATLAB conversion for Loop Recognition algorithm.


Week Ending - 09/09/05

Started translation of MATLAB .m scripts to 'C' for coronal loop recognition algorithm.

Read Mike Wheatland's paper on 'Parallel Construction of Nonlinear Force-Free Fields'.

Receieved reply from Tahar Amari about 'XTRAPOL'. He is willing to let MSSL use the software tool.

Objectives for next week

Continue with MATLAB -> 'C' translation.

Start updating AstroGrid components on 'msslxx' to version 1.1.


Week Ending - 02/09/05

Created script to compare revision of AstroGrid .war files installed on 'msslxx' against the latest in the AstroGrid repository.

Installed Linux (Red Hat 9.0) on laptop (now dual-boot) and attempted to install MATLAB.

Objectives for next week

Start manual translation of MATLAB files to 'C' if MATLAB license unresolved.


August 2005

Week Ending - 26/08/05

Experimented with open-source MATLAB clones (Octave/Sci-lab).

Investigated XTRAPOL magnetic extrapolation tool.

'mssll9' Linux box (running Red Hat 7.0) replaced with 'msslli' (running Fedora Core)

Objectives for next week

Attempt MATLAB installation on laptop (requires Linux installation).


Week Ending - 19/08/05

Experimented with loaned MATLAB disks. Licensing problem initially but now resolved.

Updated MagneticFieldExtrapolation template to include Thomas Wiegelmann's magnetogram preprocessing technique.

Objectives for next week

Continue with MATLAB installation.

On holiday (Wed - Fri)


Week Ending - 12/08/05

Completed first draft of Magnetic Field Extrapolation template.

Attended informal meeting with Elizabeth and Len to discuss progress with Magnetic Field Extrapolation algorithm.

Objectives for next week

Complete Helicity template.

Continue with Loop Recognition algorithm (MATLAB disks permitting).


Week Ending - 05/08/05

Experimented with other TRACE FITS images using Steger's curvilinear detection method. Results posted on SDO Twiki under LoopRecognition.

Completed first draft of Loop Recognition template and started work on the Magnetic Field Extrapolation template.

Objectives for next week

Request MATLAB disks in order to run/experiment with 'Improved Curve Tracing' .m source code provided by Karthik Raghupathy.

Complete Magnetic Field Extrapolation template and begin work on Helicity template.


July 2005

Week Ending - 29/07/05

Obtained first results from Carsten Steger's curvilinear detection algorithm, using OCM lt051998.fits file as input.

Objectives for next week

Complete Loop Recognition template(with reference to investigation into Steger's algorithm).

Start Magnetic Field Extrapolation template.

Write-up investigation into Steger's algorithm.

Add improvements to Steger's algorithm + RADON transform.


Week Ending - 22/07/05

Continued with Loop Recognition tool. Started integrating Carsten Steger's curvilinear detection algorithm for local curve detection.

Objectives for next week

Contact Karthik Raghupathy to check if there is Open Source code available for the algorithm described in his paper 'Improved Curve Tracing in Images'.

Continue with algorithm templates.


Week Ending - 15/07/05

Continued work on Loop Recognition tool based on RADON transform.

Objectives for next week

Start to integrate RADON transform into Loop Recognition tool and continue filling out Loop Recognition template.


Week Ending - 08/07/05

Started coding magnetogram-independent Loop Recognition tool based on RADON transform + improved curve tracing work of Raghupathy et al, and incorporating OCM pre-processing techniques. Idea is to compare results/processing times with OCM.

Curvelet transform to be tried in place of RADON transform at a later date.

Objectives for next week

Continue coding of Loop Recognition tool.


Week Ending - 01/07/05

Investigated and summarised possible alternative Loop Recognition methods.

Objectives for next week

Start coding some of the existing Loop Recognition techniques in 'C' for comparison/demo purposes.

Contact Marc DeRosa about his latest Loop Extrapolation/Magnetic Field extrapolation research at LMSAL.


June 2005

Week Ending - 24/06/05

Attended one day AIA meeting at UCL.

Started investigating possible alternative methods for Loop Recognition algorithm.

Objectives for next week

Review and write-up feasible alternative Loop Extrapolation techniques.


Week Ending - 17/06/05

Reviewed OCM and contemporary Magnetic Extrapolation techniques ahead of next week's AIA meeting at UCL.

Objectives for next week

Attend AIA meeting at UCL.

Discuss MSSL algorithm templates.


Week Ending - 10/06/05

Compared pros/cons of the different Magnetic Extrapolation models available.

Added PFSS Magnetic Extrapolation algorithm steps and Magnetic Extrapolation model summary to eSDO Twiki.

Objectives for next week

Review Helicity literature gathered.

Compile questions for Carl Shrijver's forthcoming visit to UCL.


Week Ending - 03/06/05

Steps from PFSS Magnetic Extrapolation algorithm extracted. Ready for addition to Twiki.

Successfully ran PFSS package from IDL commandline. Small modification to file 'map_set.pro' required.

Objectives for next week

Add Magnetic Extrapolation steps to eSDO Twiki.

Continue with Helicity literature search.

Compile questions for Carl Shrijver's visit to UCL on June 22nd.

Look at algorithm templates.


May 2005

Week Ending - 27/05/05

Investigated "Out of Memory" error message that occurs when copying large files through an AstroGrid workflow.

Continued analysis of Marc deRosa's Solarsoft code for Magnetic Extrapolation algorithm.

Objectives for next week

Add Magnetic Extrapolation algorithm "recipe" to Twiki.


Week Ending - 20/05/05

Completed first draft of AstroGrid Tutorial.

Started looking at IDL Solarsoft PFSS algorithm.

Objectives for next week

Continue investigating PFSS algorithm, extract essential steps and adding to Twiki.


Week Ending - 13/05/05

Got existing workflows running under the eSDO AstroGrid set-up.

Added AstroGrid Tutorial to the eSDO Twiki (95% complete).

Objectives for next week

Complete AstroGrid Tutorial (add screenshots).

Return to MSSL eSDO algorithm work.


Week Ending - 06/05/05

Attended eSDO consortium meeting at Birmingham University.

Started looking at running workflows through eSDO (msslxx) portal.

Objectives for next week

Get existing workflows running through eSDO portal and tidy-up webapps.

Put together an AstroGrid Tutorial on the eSDO Twiki.


April 2005

Week Ending - 29/04/05

CEA filecopy application running thru' portal workflow.

Prepared MSSL eSDO algorithm slideshow and backup AstroGrid presentation.

Objectives for next week

Attend eSDO consortium meeting at Birmingham University.

Contact AG developers about problems in copying large files to MySpace through a workflow.


Week Ending - 22/04/05

Assisted with Exist (XML)->MySQL data conversion on msslxv (Solar-B) server.

Continued with CEA filecopy application on AstroGrid workflow.

Objectives for next week

Complete CEA filecopying application for Birmingham demonstration.

Prepare slides for B'ham meeting.


Week Ending - 15/04/05

Integrated Cadairidris AG portal with msslxx PAL.

Continued Helicity literature review.

Objectives for next week

Create msslxx -> MySpace file copy application/workflow on Cadairidris.

Start preparing consortium meeting presentation.

Add Helicity literature comments to twiki.


Week Ending - 08/04/05

Latest PAL (astrogrid-pal-skycatserver-0.9-b010pl.war) installed and configured on msslxx to connect to MySQL. Configuration changes added to PhaseADemoPAL area of SDO twiki.

Reviewed Solar Helicity literature.

Objectives for next week

Integrate new PAL with Zhumulangma and Cadairidris AG portal workflows.

Add Helicity literature summary to twiki.


Week Ending - 01/04/05

Attended quarterly progress report with Len and Elizabeth.

Reviewed and added Magnetic Field Extrapolation literature + AstroGrid portal workflow comments to eSDO twiki.

Objectives for next week

Add and configure latest PAL on msslxx and delete old versions.

Continue with AstroGrid workflow exercise.

Continue Solar Helicity literature search.


March 2005

Week Ending - 25/03/05

Attended all-day AstroGrid training course.

Continued Magnetic Loop Extrapolation literature search.

Objectives for next week

Drop in new PAL on msslxx and check if SQL-ADQL snag fixed.

Add Magnetic Loop literature to Twiki.

Complete cea-commandline exercise.


Week Ending - 18/03/05

Managed to get AG portal workflow to run queries on msslxx MySQL database. Started work on a 'Groovy' script designed to copy selected files to MySpace on the AG portal.

Supported Stanford demo.

Continued search for Magnetic Loop Extrapolation literature.

Objectives for next week

Attend Kevin Benson's AG training course.

Complete Groovy script exercise.

Install new PAL on msslxx (to correct SQL->ADQL incompatibility with Zhumulangma AG portal).

Read Magnetic Loop Extrapolation literature gathered so far.


Week Ending - 11/03/05

Received reply from Marc de Rosa (Lockheed Martin) about current state of his Loop Recognition algorithm, plus code samples.

Started literature search for Magnetic Field Extrapolation algorithm.

Continued with AG portal - msslxx PAL integration. PAL now recognises registered application but query fails with SQL syntax error. Possible cause is SQL->ADQL conversion at AG portal end.

Objectives for next week

Investigate SQL->ADQL conversion (compare with PAL .jsp SQL->ADQL conversion)

Add 'copy file to MySpace' script to AG portal workflow.

Continue Magnetic Field Extrapolation literature search.


Week Ending - 04/03/05

Removed PAL and TOMCAT installations from home (ms2) directory on msslxx. Sole PAL installation on msslxx is under 'griduser'.

Attended AstroGrid meeting to discuss plans for mssly5.

Looked at PAL-CEA integration. msslxx PAL CEA services now registered on AstroGrid portal, compatible with the latest registry standard (0.10).

Discussed msslxx backup with Paul. He has backup scripts available that we can use as a template and run as a cron job. We need to decide upon a backup stategy and get funding for the DVD medium.

Objectives for next week

Continue with PAL-CEA integration. Plan is to create two workflows on the AG portal to:

* Query the msslxx MySQL database and return the results in VOTABLE format.

* Copy files from a given pathname to AG portal 'MySpace'.

Continue search for Magnetic Loop Extrapolation literature.


February 2005

Week Ending - 25/02/05

Moved downloaded Soho MDI files (30th Dec 01 - 2nd Feb 02) from my home directory to griduser datastore directory on msslxx server and created a new table ('mdi') under the esdo MySQL database with links to these files. The new table has the same schema as the 'trace' table. Both tables can be queried from the 'griduser' PAL ADQL/SQL query form.

Attempted to track down documentation describing AstroGrid PAL architecture, but such a document doesn't exist at the moment. Reviewing the source code Astrogrid CVS Repository remains the best way of understanding the system.

Experimented with IDL Trace image and median filtering - the first step used in the Automated Loop Recognition method.

Objectives for next week

Remove PAL and TOMCAT from home directory (ms2) on msslxx.

Start installation of CEA on the 'griduser' PAL.

Look at installing DVD writer on msslxx server for backups.

Begin search for Magnetic Loop Extrapolation papers.

Investigate how to access multiple databases under one PAL.

Attend informal AstroGrid meeting on Tuesday with Elizabeth, Kevin et al.


Week Ending - 18/02/05

Configured new PAL installation on msslxx (installed under username 'ms2') to connect to MySQL database. Able to run queries from the PAL ADQL/SQL query form.

Received Jong ("Jake") Kwan Lee's evaluation copy of 'Oriented Connectivity Method' Loop Recognition software. Runs successfully on IDL Virtual Machine (version 6.1 for Windows) installed on laptop. Results look promising but application crashes when automatic OCM is run.

Downloaded MDI 96m definitive data Magnetograms from the Stanford University archives to ~ms2/datastore/soho/mdi area on the msslxx server. Data stored in 5 sub-directories, one for each week between 30th Dec 2001 - 02 Feb 2002. These dates correspond with those of the TRACE files currently stored under the 'griduser' area on msslxx.

Updated eSDO Twiki with OCM evaluation software and documentation.

Stanford Soho MDI archive url: http://soi.stanford.edu/magnetic/index5.html

Objectives for next week

Investigate the consequences of running multiple PALs on a single server, with each accessing the same database/table.

Add new table to MySQL database with links to the MDI magnetogram data stored under 'ms2' (see above).

Investigate/resolve OCM application crash.

Add to PAL installation write-up on eSDO Twiki.


Week Ending - 11/02/05

Discussion with Elizabeth about AIA/HMI Quicklook and high-level dataproducts.

Installed PAL under home directory on 'msslxx' server. Installation passed all self-tests apart from 'SkyNode' check. Wrote small webapp that allows browser to access MySQL on the server through TOMCAT. Experimented with accessing MySQL through the PAL ADQL/SQL form.

Attempted to retrieve MDI format files from SOHO archives (RAL et al) to populate new PAL datacentre.

Contacted by e-mail Dr Allen Gary at Nasa's Marshall Space Flight Centre regarding Automated Loop Recognition software.

Objectives for next week

Resolve PAL - MySQL connection snag.

Populate new PAL datacentre with relevant MDI files.

Experiment with Automated Loop Recognition IDL software received from Dr "Jake" Lee.


Week Ending - 04/02/05

Reviewed Stanford JSOC HMI/AIA documents to determine how/where the U.K eSDO algorithms fit into the HMI/AIA data product pipelines, and how they are used to produce QuickLook and other data products. E-mailed Rick Bogart at Stanford for clarification of the subject.

Links to the reviewed documents added to the eSDO Wiki 'Quicklook products' forum.

Continued search for Loop Recognition literature/software. Found references to two automated loop tracing algorithms:one by Marc de Rosa of University of Colorado (?), the other by Allen Gary of Nasa's Marshall Space Flight Centre. Cannot find any literature supporting de Rosa's method, but a paper explaining the work of Gary et al - 'Automated Detection of Solar Loops by the Oriented Connectivity Method' is available online.

Links relating to work of de Rosa and Gary added to the eSDO Wiki 'Loop Recognition' forum.

Experimented with downloaded IDL code for a Semi-Manual Method of Loop Tracing developed by Jong K.Lee (one of the co-authors on Allen Gary's paper). Method provides the best curve fit for points entered manually on an image. Method is quick and gives reasonable results, but it is interpolative, and eSDO project requires an extrapolative means of loop tracing.

Objectives for next week

Install PAL on 'msslxx' server using MDI files as data source.

Discussion with Elizabeth about QuickLook and other data products.


January 2005

Week Ending - 28/01/05

Created and Installed test XML file on msslxx server for AstroGrid demo with Stanford. File based on the VOResource

schema.

Attended Friday's AstroGrid demo/telecon with Stanford University.

Continued experimentation with IDL SolarSoft routines.

Objectives for next week

Install PAL for Solar B project on 'msslxx' server using same set-up methods as for the eSDO PAL.

Week Ending - 21/01/05

Complete set of files extracted from the RAL TRACE archives for 30Dec2001 - 02Feb2002 and copied to griduser area on 'msslxx'.

Sat in on AstroGrid installation on msslxx server.

Downloaded and experimented with 'Oxygen' XML IDE on laptop. Created simple XML file for the AstroGrid demo based on the core properties of the VOResource-0.10.xsd schema. File called 'demoastrogrid.xml' and copied to msslxx server.

Objectives for next week

Assist with AstroGrid demo set-up.


Week Ending - 14/01/05

Investigated possible use of the Curvelet transform for the 'Loop Recognition' algorithm on the eSDO project. Findings reported in the 'Development - SolarAlgorithms' section of the Wiki.

Experimented with IDL under the SolarSoft environment.

Personal account created on eSDO server 'msslxx'. Started to populate griduser 'trace' sub-directory with TRACE data downloaded from the RAL TRACE archives. Slow progress as attempts to unzip the downloaded .zip files are frequently unsuccessful, usually as a result of the download being terminated prematurely.

Objectives for next week

Populate msslxx 'trace' sub-directory with all Jan 2002 data from the RAL TRACE archives. Assist with building AstroGrid on msslxx.

Investigate the 'a Trous' variation of the Wavelet transform, suggested by David Williams as an alternative starting point for the Loop Recognition algorithm.


Edit | Attach | Watch | Print version | History: r63 < r62 < r61 < r60 < r59 | Backlinks | Raw View | More topic actions
Topic revision: r63 - 2008-03-06 - MikeSmith
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback