Tags:
create new tag
view all tags

Install NetDRMS 07/12/07

Alpha release from Rick Bogart
7 Dec 2007

Also see InstallDRMS, InstallNetDRMS, InstallNetDRMS061207

Release

Customization

Customization required for MSSL system

  1. Adapt hardcoded paths for MSSL file system:
    • In $JSOCROOT/base/sums/apps/driven_svc.c, change sprintf(md5filter, "/home/production/cvs/jsoc/bin/%s/md5filter" to sprintf(md5filter, "/disk/d2/griduser/drms/bin/%s/md5filter"
    • In $JSOCROOT/base/sums/apps/sum_rm.c, change #define CFG_FILE "/home/production/cvs/JSOC/base/sums/apps/data/sum_rm.cfg" to #define CFG_FILE "/disk/d2/griduser/drms/base/sums/apps/data/sum_rm.cfg"
    • In $JSOCROOT/base/sums/libs/api.tape.h change #define LIBDEVFILE "/home/production/cvs/JSOC/base/sums/apps/data/libdevfile.cfg" to #define LIBDEVFILE "/disk/d2/griduser/drms/base/sums/apps/data/libdevfile.cfg"
    • Ignore hardcoded paths in base/sums/scripts for now
  2. Changes to $JSOCROOT/make_basic.mk required for 1) user installation of Postgres and 2) use of gcc instead of icc:
    • Change PGIPATH = /usr/include/pgsql to PGIPATH = /disk/d2/griduser/pgsql/include
    • Change COMPILER = icc to COMPILER = gcc
    • Change F77 = ifort to F77 = f77
    • Change $(SERVEREXE): LL_TGT := $(LL_TGT) -lpq to $(SERVEREXE): LL_TGT := $(LL_TGT) -L/disk/d2/griduser/pgsql/lib -Wl,-E,-rpath=/disk/d2/griduser/pgsql/lib -lpq
    • Change $(MODEXE): LL_TGT := $(LL_TGT) -lpq to $(MODEXE): LL_TGT := $(LL_TGT) -L/disk/d2/griduser/pgsql/lib -Wl,-E,-rpath=/disk/d2/griduser/pgsql/lib -lpq
  3. Changes to $JSOCROOT/base/sums/apps/Rules.mk:
    • Change LL_TGT_$(d) := -lecpg -lpq to LL_TGT_$(d) := -lecpg -L/disk/d2/griduser/pgsql/lib -Wl,-E,-rpath=/disk/d2/griduser/pgsql/lib -lpq
    • (Step from Karen Tian suggested on 31/10/07) Change tenerife.tuc.noao.edu to msslxx.mssl.ucl.ac.uk

Customization advised on JSOC "Installing NetDRMS" instructions

  1. In src/base/drms/apps/serverdef.h ($JSOCROOT/base/drmsapps/serverdef.h), change hmidb to msslxx.mssl.ucl.ac.uk; change jsoc to name of NetDRMS database (remains jsoc)
  2. In src/base/sums/libs/api/SUM.h ($JSOCROOT/base/sums/libs/api/SUM.h) change the following lines:
    #ifdef SUMDC
    #define SUM_VERSION_NUM (1.0)
    #define SUMSERVER "dcs0.Stanford.EDU"
    #define SUMDB "dcs0"
    #define TAPEVIEWERNAME "t50view"
    #define SUM_STOP_NOT "/usr/local/logs/SUM/SUM_STOP_NOT"
    #else
    #define SUM_VERSION_NUM (1.0)
    #define SUMSERVER "d00.Stanford.EDU"
    #define SUMDB "hmidb"
    #define TAPEVIEWERNAME "t120view"
    #define SUM_STOP_NOT "/usr/local/logs/SUM/SUM_STOP_NOT"
    #endif
    to
    #define SUM_VERSION_NUM (1.0)
    #define SUMSERVER "msslxx.mssl.ucl.ac.uk"
    #define SUMDB "jsoc"
    #define TAPEVIEWERNAME "t50view"
    #define SUM_STOP_NOT "SUM_log_dir/SUM_STOP_NOT"
  3. In src/base/sums/apps/sum_svc.c ($JSOCROOT/base/sums/apps/sum_svc.c), comment out both ifndef SUMNOAO sections
  4. In src/base/sums/libs/pg/SUMLIB_PgConnect.pgc ($JSOCROOT/base/sums/libs/pg/SUMLIB_PgConnect.pgc), change dbuser and dbhost strings to production and msslxx.mssl.ucl.ac.uk respectively

Compilation

  1. Be sure to set $JSOCROOT to new NetDRMS directory
  2. make clean
  3. ./configure
  4. make
  5. make sums - SUCCESS!

SUMS / DRMS servers

Production user and Postgres Checks

  1. Add user 'production' to postgreSQL jsoc db:
    • jsoc=# create role production;
      jsoc-# alter user production with password '*********';
      jsoc-# grant jsoc to production;
      jsoc-# alter user production LOGIN;
    • Note: postmaster stop / start:
      pg_ctl stop -W -D /disk/d2/griduser/pgsql/data/ & and pg_ctl start -W -D /disk/d2/griduser/pgsql/data/
    • Create ~production/.pgpass with format "host.with.domain.com:*:*:username:password" and chmod 0600
  2. Run masterlists:
    -bash-2.05b$ masterlists dbuser=production JSOC_DBHOST=msslxx JSOC_DBNAME=jsoc namespace=production nsgrp=user
  3. Insert production user and namespace into admin.sessions:
    % psql jsoc -U postgres
    jsoc=# insert into admin.sessionns values ('production', 'production'); Success!
  4. Ensure all db tables have been created
    • get create_tables.sql from http://www.mps.mpg.de/projects/seismo/GDC1/Assets/sql_files/create_tables.sql
    • bash-2.05b$ psql -d jsoc -f create_tables.sql
    • NOTE: had to create table sum_partn_avail by hand (cut and paste from create_tables.sql); above script did not create this table, but it also did not throw an error to say table creation had failed.
    • NOTE 2: make sure that user production owns all SUMS tables and sequences

SUMS server initialization, execution, series commands

  1. Insert values into sum_partn_avail: jsoc=# insert into sum_partn_avail (partn_name, total_bytes, avail_bytes, pds_set_num) values ('/disk/d3/production', 48593149000, 33347219000, 0); - SUCCESS
  2. Run SUMS server as user production (run as background process): ./bin/linux_ia32/sum_svc jsoc &.
    • SUCCESS Message appears as
      Please wait for sum_svc and tape inventory to initialize...
      sum_svc now available
    • Relevant processes appears as
      product 26964 0.0 0.0 4440 1340 pts/4 S 14:44 0:00 ./bin/linux_ia32/sum_svc jsoc
      product 26968 0.0 0.0 4188 1328 pts/4 S 14:44 0:00 sum_rm jsoc 2007.12.07.144410
      griduser 26969 0.0 0.1 17816 4312 ? S 14:44 0:00 postgres: production jsoc 128.40.71.12(38300) idle
      griduser 26976 0.0 0.1 17816 4632 ? S 14:44 0:00 postgres: production jsoc 128.40.71.12(38301) idle
  3. Try running create_series:
    • FAIL: 1st attempt
      create_series sngb_rot30N_48x20.jsd
      • using file sngb_rot30N_48x20.jsd from Ray Burston with seriesname owner changed to production and Tapegroup changed to 0
      • Error message:
        [production@msslxx TestDRMSFiles]$ create_series sngb_rot30N_48x20.jsd
        Creating new series 'production.SNGB_rot30N_48x20'...
        query failed: ERROR: invalid byte sequence for encoding "UTF8": 0xc56b
        Error at /disk/d2/griduser/DRMS/base/libs/db/server/db_postgresql.c, line 809: Query 'insert into drms_series(seriesname, description, author, owner, unitsize, archive,retention, tapegroup, primary_idx, created) values (?,?,?,?,?,?,?,?,?,LOCALTIMESTAMP(0))' failed.
        drms_insert_series(): failed to insert series production.SNGB_rot30N_48x20.
        Failed to create series.
    • SUCCESS: 2nd attempt
      • FIX: in sngb_rot30N_48x20.jsd, shorten Description to "Convection simulation" and shorten Author to "S, N, G, B"
      • Message:
        production@msslxx TestDRMSFiles]$ create_series sngb_rot30N_48x20.jsd
        Creating new series 'production.SNGB_rot30N_48x20'...,br/>NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "sngb_rot30n_48x20_pkey" for table "sngb_rot30n_48x20"
  4. Run drms_series.sql (from Karen Tian via Ray Burston) script:
    • Create "plpgsql" language in ,tt>jsoc db:
      [production@msslxx TestDRMSFiles]$ psql -d jsoc -U postgres
      jsoc=# create language plpgsql;
    • SUCCESS: [production@msslxx TestDRMSFiles]$ psql -d jsoc -f drms_series.sql
  5. Try running delete_series:
    [production@msslxx TestDRMSFiles]$ delete_series production.SNGB_rot30N_48x20
    • SUCCESS: message:
      You are about to permanently erase all metadata for the series 'production.SNGB_rot30N_48x20'.
      Are you sure you want to do this (yes/no)? yes
      I repeat: All data records from 'production.SNGB_rot30N_48x20' will be erased.
      Are you REALLY sure you want to do this (yes/no)? yes
      Removing existing series 'production.SNGB_rot30N_48x20'...
  6. Run create_series again (to create production.SNGB_rot30N_48x20) and try show_series:
    • SUCCESS:[production@msslxx TestDRMSFiles]$ show_series
      production.SNGB_rot30N_48x20

DRMS server execution

  1. Try running drms_server:
    • FAIL: [production@msslxx drms]$ drms_server
      • Error:
        [production@msslxx TestDRMSFiles]$ drms_server
        DRMS server started with pid=0, noshare=0
        env->session->db_direct = 1
        DRMS server connected to database 'jsoc' on host 'msslxx.mssl.ucl.ac.uk' as user 'production'.
        DRMS server listening on msslxx.mssl.ucl.ac.uk:38325.
        [production@msslxx TestDRMSFiles]$ suidback = 0
        Failed to get SUMID from sum_svc
        drms_open: Failed to connect to SUMS.
        SUM ALLOC failed with error code -10034.
        Failed to allocate storage unit for log files: -10034
        DRMS_HOST = msslxx.mssl.ucl.ac.uk
        DRMS_PORT = 38325
        DRMS_PID = 27085
        DRMS_SESSIONID = 6
        DRMS_SESSIONNS = production
        DRMS_SUNUM = 0
        DRMS_SUDIR = (null)
        Listening for incoming connections on port 38325.
        WARNING: DRMS server called exit.

Issues

  1. Paths still hardcoded rather than picked up from $JSOCROOT in the following files:
    -bash-2.05b$ grep -r "home/production" .
    ./base/sums/apps/driven_svc.c: sprintf(md5filter, "/home/production/cvs/jsoc/bin/%s/md5filter",
    ./base/sums/apps/sum_rm.c:#define CFG_FILE "/home/production/cvs/JSOC/base/sums/apps/data/sum_rm.cfg"
    ./base/sums/libs/api/tape.h:#define LIBDEVFILE "/home/production/cvs/JSOC/base/sums/apps/data/libdevfile.cfg"
    ./base/sums/scripts/sum_start:#/home/production/cvs/JSOC/base/sums/scripts/sum_start
    ./base/sums/scripts/sum_start_dc:#/home/production/cvs/JSOC/base/sums/scripts/sum_start
    ./base/sums/scripts/sum_stop:#/home/production/cvs/JSOC/base/sums/scripts/sum_stop
    ./base/sums/scripts/sum_stop_dc:#/home/production/cvs/JSOC/base/sums/scripts/sum_stop
    ./base/sums/scripts/sumlook.pl:#/home/production/cvs/JSOC/base/sums/scripts/sumlook.pl
    ./base/sums/scripts/t120view:$OURPATH = "/home/production/cvs/JSOC/base/sums/scripts/t120view"; #!!this program
    ./base/sums/scripts/t50view:$OURPATH = "/home/production/cvs/JSOC/base/sums/scripts/t50view"; #!!this program
    ./base/sums/scripts/tapearc_do:/home/production/cvs/jsoc/bin/linux_x86_64/tapearc -v jsoc >>& /tmp/tapearc_cron.log
    ./base/sums/scripts/tapearc_do_dcs1:/home/production/cvs/jsoc/bin/linux_x86_64/tapearc -v jsocdc >>& /tmp/tapearc_cron.log
  2. In /etc/sudoers, added line production [hostname]:NOPASSWD:/bin/chown, /bin/chmod. So far seems to have no effect, and firewall to msslxx has been closed while line is in place. (ECA 12:06 07/12/07) Edit: now working with this /etc/sudoers line in place; firewall hole is still closed, however, so must find solution with both firewall open and line in place.

-- ElizabethAuden - 07 Dec 2007

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | More topic actions
Topic revision: r2 - 2007-12-09 - ElizabethAuden
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback