Tags:
create new tag
view all tags

Network Latency Tests


Proposal for Testing Network Latencies for eSDO
08 September 2005
Rick Bogart and Elizabeth Auden

Aim: establishing baselines for network latency involved in data transfer under various protocols. This information is required for the specification of both the optimal sizes for the eSDO data cache elements and the overall architecture for an interoperating system involving the JSOC capabilities for efficient data management at the two (or more) sites.

Transfer protocols to be investigated include simple higher-level ones such as ftp, wget, and rsync, and especially lower-level ones including gridFTP, nfs, and raw TCP data sockets handled through dedicated clients. The tests will require accounts for Elizabeth at MSSL & Stanford (existing) and UCL and/or RAL on designated Linux machines, and accounts for Rick at Stanford (existing) and MSSL plus UCL and/or RAL on the same machines, if possible. They will also require temporary access to approximately 100 GB of scratch disk space at each site. We will also need cooperation of the system administrators in opening any firewalls for the designated hosts, ports and services for the tests.

I. Transfer mechanisms

  1. non-interactive scp
    • login to Stanford machine: scp files to MSSL
    • login to Stanford machine: scp files from MSSL
    • login to MSSL:scp files to Stanford
    • login to MSSL: scp files from Stanford
    • N.B: These may require .rhosts type authentication for non-interactive transfers
  2. ftp
    • login to Stanford machine: ftp files to MSSL
    • login to Stanford machine: ftp files from MSSL
    • login to MSSL: ftp files to Stanford
    • login to MSSL: ftp files from Stanford
    • N.B. These may require access to anonymous ftp if use of plain-text .netrc files is to be avoided for non-interactive transfers
  3. gridFTP
    • login to Stanford machine: gridftp files to MSSL
    • login to Stanford machine: gridftp files from MSSL
    • login to MSSL: gridftp files to Stanford
    • login to MSSL: gridftp files from Stanford
  4. wget
    • login to Stanford machine: wget files to MSSL
    • login to Stanford machine: wget files from MSSL
    • login to MSSL: wget files to Stanford
    • login to MSSL: wget files from Stanford
  5. rsync
    • test rsync for speed in updating files that exist at both MSSL and Stanford
    • test rsync for speed in updating directory structures existent at MSSL and Stanford
    • test rsync for ability to update dynamic directory structures (in context of DRMS) at MSSL and Stanford
  6. direct access
    • test comparative network latency for file seeks, reads, and writes for files mounted via a standard protocol (nfs) from remote sites
    • test latencies for selected data transfers using tcp sockets

II. Test Requirements

  1. Remote accounts
    • Accounts for Elizabeth at Stanford (done), UCL and / or RAL
    • Accounts for Rick at MSSL, UCL, RAL if possible
  2. Test Data . up to 100 GB in scratch space for bulk transfers
  3. Test locations
    • Stanford
    • MSSL
    • UCL
    • RAL
  4. Test scripts to automatically (and non-interactively) initiate transfers for test data and store metrics for speed and timestamps
  5. Required programs (with opened ports) on test machines: scp, ftp, gridftp, wget, rsync

-- ElizabethAuden - 29 Sep 2005

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | More topic actions
Topic revision: r2 - 2006-05-10 - ElizabethAuden
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback