Tags:
create new tag
view all tags

Ingesting Simulated Data into DRMS

09/01/08 HELAS workshop in Freiburg

Summary

Working with Rick and Ray on DRMS / SUMS data ingestion at the HELAS workshop in Freiburg this week. I'm currently stuck on ingesting simulated data into series sha.SNGB_rot30N_48x20 (failure at step 6 below).

Log

(Note: all work done as user production on msslxx.mssl.ucl.ac.uk)

1. create sha namespace in msslxx DRMS:
-bash-2.05$ masterlists dbuser=production JSOC_DBHOST=msslxx JSOC_DBNAME=jsoc namespace=sha nsgrp=user

2. Insert namespace sha into admin.sessionns for user production:
% psql jsoc -U postgres
jsoc=# insert into admin.sessionns values ('production', 'production');

3. create series sha.SNGB_rot30N_48x20
-bash-2.05$ create_series sngb_rot30N_48x20.jsd
-bash-2.05$ show_series
production.NOAA_ActiveRegions
sha.SNGB_rot30N_96x20
sha.SNGB_rot30N_48x20

4. scp'd 9 fits files of simulated data from eauden@solarport.stanford.edu:/tmp13/dali/TEMP.1222.2, giving
bash-2.05b$ ls /disk/d2/griduser/TestDRMSFiles/SNGB
Ei.1222.2r.fits Nel.1222.2r.fits Rho.1222.2r.fits T.1222.2r.fits Uy.1222.2r.fits
Gam.1222.2r.fits P.1222.2r.fits sngb_rot30N_48x20.jsd Ux.1222.2r.fits Uz.1222.2r.fits

5. compiled ingest_sngb.c from Rick

6. FAIL: attempt to ingest data from above 9 fits files into DRMS using ingest_sngb:
[production@msslxx ingest]$ ./ingest_sngb dir=/disk/d2/griduser/TestDRMSFiles/SNGB &
[1] 18593
[production@msslxx ingest]$ Failed to get SUMID from sum_svc
drms_open: Failed to connect to SUMS.
SUM ALLOC failed with error code -10034.
Error: could not create record in data series "sha.SNGB_rot30N_48x20"
WARNING: DRMS server aborting in 2 seconds...
WARNING: DRMS server called exit.
[1]+ Exit 1 ./ingest_sngb dir=/disk/d2/griduser/TestDRMSFiles/SNGB

Ray and Rick have advised several checks to see where the SUMS failure might be happening, but we have not been able to identify the cause.

1. sum_svc is running: [production@msslxx ingest]$ ps aux | grep sum_svc
product 17760 0.0 0.0 4576 1360 pts/4 S 09:40 0:00 sum_svc jsoc

2. the SUMS partition area has been declared in table sum_part_avail:
jsoc=> select * from sum_partn_avail;
partn_name | total_bytes | avail_bytes | pds_set_num
---------------------+-------------+-------------+-------------
/disk/d3/production | 48593149000 | 31404066324 | 0
(1 row)

3. the following tables and sequences have been created by postgres user "production" in database "jsoc":
jsoc=> \d
List of relations
Schema | Name | Type | Owner
------------+------------------------+----------+------------
production | drms_keyword | table | production
production | drms_link | table | production
production | drms_segment | table | production
production | drms_series | table | production
production | drms_session | table | production
production | drms_sessionid_seq | sequence | production
production | noaa_activeregions | table | production
production | noaa_activeregions_seq | sequence | production
public | sum_ds_index_seq | sequence | production
public | sum_file | table | production
public | sum_group | table | production
public | sum_main | table | production
public | sum_open | table | production
public | sum_partn_alloc | table | production
public | sum_partn_avail | table | production
public | sum_seq | sequence | production
public | sum_tape | table | production
(17 rows)

-- ElizabethAuden - 09 Jan 2008

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | More topic actions
Topic revision: r1 - 2008-01-09 - ElizabethAuden
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback