Tags:
create new tag
view all tags

Problems discovered during testing

This list documents issues discovered during the testing of the REAPER L2 chain. The current status of the ReaperProducts is maintained on that page.

There are also ReaperIssues which were noted during development. These need to be checked to find out if they are problems.

Documents that may need to be updated can be found in here msslc6:/home/djb/00_REAPER_docs/. Take a copy, make a small change, and put it back here with a suffix describing the change. DJB will merge it in Word and then put the updated master copy back.

During testing, capture how you test a field and put that information into the test document. That way, we write it as we go along.

If you spot something that is correct according to the DPM accepted at CDR but could be improved, then note it as ReaperImprovements.

Renumbering of fields and sizes in documentation

Symptom
After the "L2 output cleanups" task (see below), many fields (i.e. netCDF variables) have been added and removed.
Diagnosis
When testing and L2 product changes are considered completed, L2 fields need renumbering in documents. Also, the byte size subtotals and totals need to be recalculated. This applies to tables in the documents and to paragraphs describing each field. The affected documents are (at least) Reaper-ProdSpec-v....docx and Reaper-IODD-v....docx.
Owner
Open (put initials here once claimed)

Corrective Steps

  1. TBD

L2 output cleanups (S)GDR

Symptom
Some variables are not filled, some are not at the right cadence, general confusion
Diagnosis
Product designer needs shooting
Owner
CD. Completed 24/8/2012 except for renumbering in documents (see task above).

Corrective Steps

Note: For all of these steps, the code needs modifying and the IODD and Product Spec documents need modifying in parallel.
  1. Remove skip_count
  2. Remove f_HRB_loc_valid
  3. Remove f_time_valid (Other changes made: also removed from meteo product)
  4. Remove P_Attitude
  5. Remove f_loc_valid
  6. Remove Altitude_stdev
  7. Remove map_alt_valid
  8. Remove onboard_range_bitmap
  9. Remove ocean_range_C
  10. Remove ocean_agc_C
  11. Remove ice2_range_C
  12. Remove ice2_agc_C
  13. Split mean_quadratic_error into mean_quadratic_error_ice2 and mean_quadratic_error_ocean. Make the internal variables (j_MQE_ocean, j_MQE_ice2) double rather than long int to preserve precision for conversion to units of 10^4 counts on output.
  14. Remove Alt_wnd
  15. Fill CG_Rng_C from d_COG_C in internal block H15
  16. Make ice2_le_width 20Hz
  17. Make ice2_1st_te_slope 20 Hz
  18. Make ice2_2nd_te_slope 20 Hz
  19. Remove L_inst_rng_C
  20. Remove L_Dop_C
  21. Remove flagging of slopes application and errors from the 1Hz flag and create new 20 Hz booleans using bytes next to the offset_ variables. f_slope_applied and f_slope_error. Fill f_slope_applied from at_RA_RP_H12_TOL[i_HRB].i_slope_corrected. Details of changes made:
    1Hz bit COR_APP_SLOPE (in t_RA_RP_L5_CORR.uj_f_corr_applied) changed to spare. Uses of it are replace by the existing boolean at_RA_RP_H12_TOL[i_HRB].i_slope_corrected. This boolean is written to L2 product in the 20Hz netCDF variable f_slope_applied.
    1Hz bit COR_ERR_SLOPE (in t_RA_RP_L5_CORR.uj_f_corr_error) changed to spare. Uses of it are replace by a new boolean at_RA_RP_H12_TOL[i_HRB].i_f_slope_error. This boolean is written to L2 product in the 20Hz netCDF variable f_slope_error.
    The related map_slope_valid field was also removed from the product.
  22. Create new variable elevation_ice1 = alt - ice1_range. To be filled after ice1_range is fully corrected, so do on output.
  23. Create new variable sum_C_applied at 20 Hz. Fill it with sum_corr from L2URNGE plus the SSB if applied for that HRB.
  24. Remove H_LPTNE (Other changes made: also removed from meteo product)
  25. Remove 20 Hz versions of Tb_23_8 and Tb_36_5 as they are not used
  26. Remove Rng_Est, SWH_Est, Sig0_Est, f_HRB_deriv_valid, noise_floor
  27. agc removed from GDR. (There remains an AGC field in SGDR).
  28. Tb_23_6 typos corrected to Tb_23_8.
  29. AGG typo corrected to AGC.

Retracker Failures

Symptom
Retrackers (OCOG and sea-ice) fail on all records.
Diagnosis
The retrackers can't cope with normal ERS waveforms. They have significant power in the first few bins and the retracker attempts to retrack to that instead of the actual waveform.
Owner
DJB Working

Corrective Steps

  1. Check the DPM and update if necessary. The algorithms should only use the bins between OCOG_start_bin and OCOG_end_bin (different name for sea-ice retracker) so that these parameters can be tuned to exclude the bins that we don't want. Note, there may be an existing algorithm spec for ERS with this in (TBC).
  2. Send updated DPM copy to DJB for merging into master
  3. Update the code (this has already been done as a quick hack to allow initial testing).

Backscatter

Symptom
Backscatter calculation is incorrect (found by inspection)
Diagnosis
Description of which fields to use in the L1b product was unclear.
Owner
DJB Working

Corrective Steps

  1. Correct method for calculating backscatter is: sig_0 [dB] = Sigma0_scale [dB] + 10.0 log_10 ( amplitude [FDPU] ) + bias
  2. Check that Sigma0_scale is taken from L1b field 27 Sigma0_scale and not 39 p_ref
  3. Check against DPM and make DPM clearer (if necessary).
  4. Update code for OCOG and sea-ice retrackers. Note: Amplitude is from retracking and bias is a constant from algorithm constants.

max_variance

Symptom
max_variance is an algorithm configuration parameter and is in the characterisation structure.
Diagnosis
Misplaced; move to algorithm constants
Owner
LKG. Done. Put in CVS 17/08/11. Copies of xml files available from LKG. Master spreadsheets NOT updated.

Corrective Steps

  1. Move the parameter from characterisation to algorithm constants and update the XML files, readers and structures.

Ionospheric code only handles one filetype

Symptom
No handling of differences between GPS and model corrections
Diagnosis
Remko supplied two different sets of files, one from a NIC09 model and the other from GPS. We just treat it all the same.
Owner
LKG. Done. Put in CVS 17/08/11. Choice of GPS or model is selectable, if no selection made GPS is the default.

Corrective Steps

  1. Duplicate the Ionospheric code so that it handles 2 filetypes, one for model and one for GPS. GPS is optional but model is mandatory.
  2. Interpolate the model and fill i_Iono_C_mod and the GPS if available and fill i_Iono_C_GIM
  3. Make the range calculation prefer GIM but fall back to model if GIM is not available.

Uses of uninitialised values

Symptom
valgrind reports many uses of an "uninitialised value" which is allocated on the stack by xf_tree_path_read_integer_node_value(). No obvious problem results from these errors. However, when running under Ubuntu this libxml2 library consistently crashed with a SEGV, during j_ALGORITHM_CONSTANTS_reader(), until it was upgraded to v2.7.8. So this library may be a prime suspect if any crashes occur in future.
Valgrind Log Line Number
55 to 225
Diagnosis
TBD
Owner
Open (put initials here once claimed)

Corrective Steps

  1. TBD

Invalid memory access in XCPOL_PLD_Compute

Symptom
valgrind: Invalid read of size 1. Within XCPOL_PLD_Compute (XCPOL_PoleLocationData.c:60)
Valgrind Log Line Number
1419
Diagnosis
sscanf was used on a string which was not null-terminated, giving unpredictable results.
Owner
CD. Investigation completed 18/08/2011.

Corrective Steps

  1. Fixes applied to revision 1.2 of XCPOL_PoleLocationData.c. Marked by "TODO" comments. Should be reviewed by original author.

Invalid memory accesses in ExtractAltiDataFromPDS

Symptom
valgrind: Two invalid reads of size 1. Within ExtractAltiDataFromPDS (XCMTO_Fmo.c:812)
Valgrind Log Line Number
1461 to 1497
Diagnosis
sscanf was used on strings which were not null-terminated, giving unpredictable results.
Owner
CD. Investigation completed 18/08/2011.

Corrective Steps

  1. Fixes applied to revision 1.2 of XCMTO_Fmo.c. Marked by "TODO" comments. Should be reviewed by original author.

Invalid memory access in ExtractMeanSSP

Symptom
valgrind: Invalid read of size 1. Within ExtractMeanSSP (XCMTO_Fmo.c:535)
Diagnosis
sscanf was used on strings which were not null-terminated, giving unpredictable results.
Owner
CD. Investigation completed 18/08/2011.

Corrective Steps

  1. Fixes applied to revision 1.2 of XCMTO_Fmo.c. Marked by "TODO" comments. Should be reviewed by original author.

Invalid memory accesses reading array

Symptom
valgrind: Invalid read of size 8. Within SelectValidRadMeasCouple (RAD_MAN_INT_01.c:308)
Valgrind Log Line Number
142302
Diagnosis
An array index was incremented beyond end of array, during a loop.
Owner
CD. Investigation completed 18/08/2011.

Corrective Steps

  1. Fix applied to revision 1.2 of RAD_MAN_INT_01.c. Marked by "TODO" comments. Should be reviewed by original author.

Memory leaks reported by valgrind

Symptom
Memory leaks are listed at the end of JobOrder_test_run_s1_o1.valgrind.leaks.log.
Diagnosis
TBD
Owner
Open (put initials here once claimed)

Corrective Steps

  1. TBD

L2_name2id string lookups are slow

Symptom
The L2_name2id lookups involve a search+comparison through a table of strings. This is apparently done 21 million times and takes up to 8 seconds of total execution time. Look in to optimising this.
Diagnosis
n/a
Owner
CD. Completed on 18/08/2011.

Corrective Steps

  1. Revision 1.11 of output_SGDR.c and 1.9 of output_meteo.c no longer use any searching or comparisons. Compile time identifiers are now in an enum. These are translated to run-time IDs by a direct lookup in an array. The chain now runs 6 seconds (7%) faster.

Next issue

Symptom
TBD
Diagnosis
TBD
Owner
Open (put initials here once claimed)

Corrective Steps

  1. TBD

-- DavidBrockley - 11 Aug 2011

Topic attachments
I Attachment History ActionSorted ascending Size Date Who Comment
Unknown file formatlog JobOrder_test_run_s1_o1.valgrind.edited.log r1 manage 20364.6 K 2011-08-17 - 09:54 ChrisDolding Reaper valgrind log showing memory access errors. Default valgrind settings (i.e. no arguments)
Unknown file formatlog JobOrder_test_run_s1_o1.valgrind.leaks.log r1 manage 15326.2 K 2011-08-18 - 09:44 ChrisDolding Reaper valgrind log showing memory leaks details (valgrind --read-var-info=yes --leak-check=full --track-origins=yes ... )
Edit | Attach | Watch | Print version | History: r17 < r16 < r15 < r14 < r13 | Backlinks | Raw View | More topic actions
Topic revision: r17 - 2011-08-24 - ChrisDolding
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback