Tags:
create new tag
view all tags

eSDO Network Latency Tests

This document is also available as a pdf.


Testing Network Latencies for eSDO
10 May 2006
Elizabeth Auden

Introduction

Network tests have been undertaken to establish baselines for network latency involved in data transfer under various protocols. This information is required for the specification of both the optimal sizes for the eSDO data cache elements and the overall architecture for an interoperating system involving the JSOC capabilities for efficient data management at the two (or more) sites. Each node represents a different use case for future SDO data requirements:
  1. Primary archive (Stanford)
  2. Secondary archive (RAL)
  3. Algorithm host machine (UCL)
  4. User machine (MSSL)

Test Design

Institutions

  • MSSL, msslxx.mssl.ucl.ac.uk, linux (fedora core 2)
  • UCL, zuserver1.star.ucl.ac.uk, linux (red hat)
  • RAL, tornado.badc.rl.ac.uk, dec alpha
  • Stanford, solarport.stanford.edu, linux (red hat)

Data

  • TRACE, T171_2004Jul15-16-X.mov, 1 Apple QuickTime movie file 1.21 GB
  • Yohkoh, 1522 SXT uncompressed FITS files totalling 1.2 GB
  • SOHO-CDS, 5003 compressed FITS files totalling 1.1 GB

Protocols

Three protocols were tested using ssh with PPK encryption for automated login. GridFTP will be tested between MSSL and Stanford at a later date.

  • scp
  • sftp
  • rsync

Scripts

The network test scripts used passphraseless public / private key encryption to automate file transfers with rsync, sftp and scp over ssh. Because of the security risks with passphraseless keys, MSSL and UCL firewall holes were restricted to specific named machines for the duration of the network tests. The sftp transfers were run in batch mode to allow multiple non-interactive file transfers At MSSL, Stanford and UCL the bash time utility was used in conjuction with the append and out facilities to log the user, system and elapsed time along with CPU usage for each rsync and scp files transfer. A single time was provided for all sftp transfers executed in a single batch.

Non-Interactive PPK

First, a set of DSA-2 public and private keys were generated on msslxx for each of the 4 test machines. Each of the machines stored its private key in the ~$user/.ssh directory, and the public key for each machine was distributed to all 4 machines' ~$user/.ssh directories.

  1. %ssh-keygen -b 1024 -t dsa
  2. %enter filename: $user_$institution_id_dsa
  3. %enter passphrase: hit enter for no passphrase
  4. %re-enter passphrase: hit enter again

Repeating this process on msslxx created a pair of public and private keys of the format $user_$institution_id_dsa and $user_$institution_id_dsa.pub for each of the 4 institutions involved. Once created, keys were transferred from msslxx to the appropriate machines.

Automated, Non-Interactive Scripts

RSYNC push

For MSSL, Stanford, and UCL machines:

echo "RSYNC PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_rsync.txt ; date >> ~$user/log/esdo_$institution_timelog_rsync.txt
for file in `cat ~$user/$instrumentfiles.txt`
do
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_rsync.txt rsync -e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa" $user/DataStore/$INSTRUMENT/$file $remoteuser@$remotemachine:/home/$remoteuser/TestData/$institution/.
done
ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa $remoteuser@$remotemachine << EOD1
cd ~$remoteuser/TestData/$institution/
rm -rf *.*
EOD1

For RAL machine:

echo "RSYNC PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_rsync.txt ; date >> ~$user/log/esdo_$institution_timelog_rsync.txt
echo "RSYNC PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_rsync.txt ; date >> ~$user/log/esdo_$institution_transferlog_rsync.txt
foreach file (`cat ~$user/$instrumentfiles.txt`)
( time rsync -e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa" $user/DataStore/$INSTRUMENT/$file $remoteuser@$remotemachine:$remoteuser/TestData/$institution/. >> ~$user/log/esdo_$institution_transferlog_rsync.txt ) >>& ~$user/log/esdo_$institution_timelog_rsync.txt
end
ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa $remoteuser@$remotemachine << EOD1
cd ~$remoteuser/TestData/$institution/
rm -rf *.*
EOD1

RSYNC pull

For MSSL, Stanford, and UCL machines:

echo "RSYNC PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_rsync.txt ; date >> ~$user/log/esdo_$institution_timelog_rsync.txt
for file in `cat ~$user/$instrumentfiles.txt`
do
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_rsync.txt rsync
-e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa"
$remoteuser@$remotemachine:/home/$remoteuser/TestData/$institution/$file
$user/TestData/$remoteinstitution/
done
cd ~$user/TestData/$remoteinstitution/
rm -rf *.*

For RAL machine:

echo "RSYNC PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_rsync.txt ; date >> ~$user/log/esdo_$institution_timelog_rsync.txt 
echo "RSYNC PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_transferlog_rsync.txt ; date >> ~$user/log/esdo_$institution_transferlog_rsync.txt
foreach file (`cat ~$user/$instrumentfiles.txt`)
( time rsync -e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa" $remoteuser@$remotemachine:$remoteuser/DataStore/$instrument/$file $user/TestData/$institution/. >> ~$user/log/esdo_$institution_transferlog_rsync.txt ) >>& ~$user/log/esdo_$institution_timelog_rsync.txt
end
cd ~$user/TestData/$remoteinstitution/
rm -rf *.*

SCP: push

For MSSL, Stanford, and UCL machines:

echo "SCP PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_scp.txt ; date >> ~$user/log/esdo_$institution_timelog_scp.txt
for file in `cat ~$user/$instrumentfiles.txt`
do
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_scp.txt scp -i ~$user/.ssh/esdo_$institutionhost_id_dsa ~$user/DataStore/$INSTRUMENT/$file $remoteuser@$remotemachine:@$user/TestData/$institution
done
ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa $remoteuser@$remotemachine << EOD
cd ~$remoteuser/TestData/$remoteinstitution/
rm -rf *.*
EOD

For RAL machine:

echo "SCP PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_scp.txt ; date >> ~$user/log/esdo_$institution_timelog_scp.txt
echo "SCP PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_transferlog_scp.txt ; date >> ~$user/log/esdo_$institution_transferlog_scp.txt
foreach file (`cat ~$user/$instrumentfiles.txt`)
( time scp -e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa" $user/DataStore/$instrument/$file $remoteuser@$remotemachine:$remoteuser/TestData/$institution/. >> ~$user/log/esdo_$institution_transferlog_scp.txt ) >>& ~$user/log/esdo_$institution_timelog_scp.txt
end
ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa $remoteuser@$remotemachine << EOD1
cd ~$remoteuser/TestData/$institution/
rm -rf *.*
EOD1

SCP: pull

For MSSL, Stanford, and UCL machines:

echo "SCP PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_scp.txt ; date >> ~$user/log/esdo_$institution_timelog_scp.txt
for file in `cat ~$user/$instrumentfiles.txt`
do
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_scp.txt scp -i ~$user/.ssh/esdo_$institutionhost_id_dsa ~$user/DataStore/$INSTRUMENT/$file $remoteuser@$remotemachine:@$user/TestData/$institution/$file ~$user/TestData/$remoteinstitution/ 
done
cd ~$user/TestData/$remoteinstitution/
rm -rf *.*

For RAL machine:

echo "SCP PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_scp.txt ; date >> ~$user/log/esdo_$institution_timelog_scp.txt
echo "SCP PULL: $institution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_transferlog_scp.txt ; date >> ~$user/log/esdo_$institution_transferlog_scp.txt
foreach file (`cat ~$user/$instrumentfiles.txt`)
( time scp -e "ssh -i ~$user/.ssh/esdo_$institutionhost_id_dsa" $remoteuser@$remotemachine:$remoteuser/DataStore/$instrument/$file $user/TestData/$remoteinstitution/ >> ~$user/log/esdo_$institution_transferlog_scp.txt ) >>& ~$user/log/esdo_$institution_timelog_scp.txt
end
cd ~$user/TestData/$remoteinstitution/
rm -rf *.*

SFTP: push

For MSSL, Stanford, and UCL machines:

echo "SFTP PUSH: $institution TO $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_sftp.txt ; date >> ~$user/log/esdo_$institution_timelog_sftp.txt
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_sftp.txt sftp -o 'IdentityFile ~$user/.ssh/esdo_$institutionhost_id_dsa' -b ~$user/sftp_push_$instrument.batch $remoteuser@$remotemachine:$user/TestData/$institution
ssh -i ~$remoteuser/.ssh/esdo_$remoteinstitutionhost_id_dsa $remoteuser@$remotemachine << EOD1
cd $remoteuser/TestData/$institution/
rm *.*
EOD1

For RAL machine:

echo "SFTP PUSH: $institution TO $remoteinstitution: $instrument, #" >> ~$user/log/esdo_$institution_timelog_sftp.txt ; date >> ~$user/log/esdo_$institution_melog_sftp.txt
echo "SFTP PUSH: $institution TO $remoteinstitution: $instrument, #" >> ~user/log/esdo_$institution_transferlog_sftp.txt ; date >> ~user/log/esdo_$institution_transferlog_sftp.txt
( time sftp -o 'IdentityFile ~user/.ssh/esdo_$institutionhost_id_dsa' -b ~$user/sftp_push_$instrument.batch $remoteuser@$remotemachine:$remoteuser/TestData $institution >> ~$user/log/esdo_$institution_transferlog_sftp.txt ) >>& ~$user/log/esdo_$institutionimelog_sftp.txt
ssh -i ~$user/.ssh/esdo_$institutehost_id_dsa $remoteuser@$remotemachine << EOD1
cd $remoteuser/TestData/$institution/
rm *.*
EOD1

SFTP push batch files:

Text file listing get command plus one file per line (no path information). For example:

get s100r00.fits.Z

SFTP: pull

For MSSL, Stanford, and UCL machines:

cd ~$user/TestData/$remoteinstitution/
echo "SFTP PULL: $insitution FROM $remoteinstitution: $INSTRUMENT, #" >> ~$user/log/esdo_$institution_timelog_sftp.txt ; date >> ~$user/log/esdo_$institution_timelog_sftp.txt
/usr/bin/time -a -o ~$user/log/esdo_$institution_timelog_sftp.txt sftp -o 'IdentityFile ~$user/.ssh/esdo_$institutionhost_id_dsa' -b ~$user/sftp_pull_$instrument.batch $remoteuser@$remotemachine:DataStore/$INSTRUMENT
rm ~$user/TestData/$remoteinstitution/*.*

For RAL machine:

echo "SFTP PULL: $institution FROM $remoteinstitution: $instrument, #" >> ~$user/log/esdo_$institution_timelog_sftp.txt ; date >> ~$user/log/esdo_$institution_melog_sftp.txt
echo "SFTP PULL: $institution FROM $remoteinstitution: $instrument, #" >> ~user/log/esdo_$institution_transferlog_sftp.txt ; date >> ~user/log/esdo_$institution_transferlog_sftp.txt
( time sftp -o 'IdentityFile ~user/.ssh/esdo_$institutionhost_id_dsa' -b ~$user/sftp_push_$instrument.batch $remoteuser@$remotemachine:$remoteuser/DataStore/$instrument >> ~user/log/esdo_$institution_transferlog_sftp.txt ) >>& ~$user/log/esdo_$institutionimelog_sftp.txt
cd $user/TestData/$remoteinstitution/
rm *.*

SFTP pull batch files:

Text file listing put command plus one file per line (full local path information). For example:

put /home/eca/DataStore/CDS/s100r00.fits.Z

Results

Test Matrix

PROTOCOL PUSH/PULL INSTITUTION REMOTE INSTITUTION INSTRUMENT KB/SEC ELAPSED SEC SIZE NUMBER OF TRIALS
RSYNC PULL MSSL RAL CDS        
RSYNC PULL MSSL RAL TRACE 4336.470601 289.21 1211107245 2
RSYNC PULL MSSL RAL Yohkoh        
RSYNC PULL MSSL RAL CDS        
RSYNC PULL MSSL STANFORD TRACE 401.1485084 2948.34 1211107245 2
RSYNC PULL MSSL RAL Yohkoh        
RSYNC PULL MSSL RAL CDS        
RSYNC PULL MSSL UCL TRACE 3194.875417 370.51 1211107245 2
RSYNC PULL MSSL RAL Yohkoh        
RSYNC PULL RAL MSSL CDS 610.577447 2059.333333 1149635736 3
RSYNC PULL RAL MSSL TRACE 2223.200026 549.25 1211107245 4
RSYNC PULL RAL MSSL Yohkoh 1060.68955 1311.666667 1271676928 3
RSYNC PULL RAL STANFORD CDS 72.37772289 15514 1149635736 3
RSYNC PULL RAL STANFORD TRACE 194.2172718 6912.5 1211107245 4
RSYNC PULL RAL STANFORD Yohkoh 203.7822694 10757 1271676928 4
RSYNC PULL RAL UCL CDS 651.5660324 1882.916667 1149635742 5
RSYNC PULL RAL UCL TRACE 3777.983018 306 1211107245 4
RSYNC PULL RAL UCL Yohkoh 4859.713795 297.875 1271676928 5
RSYNC PULL STANFORD MSSL CDS 35.83666908 31328 1149635757 1
RSYNC PULL STANFORD MSSL TRACE 392.0841767 3016.5 1211107245 1
RSYNC PULL STANFORD MSSL Yohkoh        
RSYNC PULL STANFORD RAL CDS        
RSYNC PULL STANFORD RAL TRACE 187.9424629 6293 1211107245 1
RSYNC PULL STANFORD RAL Yohkoh        
RSYNC PULL STANFORD UCL CDS 52.50145758 21384 1149635757 1
RSYNC PULL STANFORD UCL TRACE        
RSYNC PULL STANFORD UCL Yohkoh 130.6821004 9503 1271676928 1
RSYNC PULL UCL MSSL CDS 523.4064667 2144.97 1149635757 1
RSYNC PULL UCL MSSL TRACE        
RSYNC PULL UCL MSSL Yohkoh        
RSYNC PULL UCL RAL CDS 335.2317614 3349 1149635757 1
RSYNC PULL UCL RAL TRACE        
RSYNC PULL UCL RAL Yohkoh        
RSYNC PULL UCL STANFORD CDS        
RSYNC PULL UCL STANFORD TRACE        
RSYNC PULL UCL STANFORD Yohkoh        
RSYNC PUSH RAL MSSL CDS 482.6726017 2468.166667 1072275345 6
RSYNC PUSH RAL MSSL TRACE 5892.994284 299.5 1211107245 4
RSYNC PUSH RAL MSSL Yohkoh 3874.416717 584.75 1271676928 4
RSYNC PUSH RAL STANFORD CDS 75.44878282 15909 1149635736 3
RSYNC PUSH RAL STANFORD TRACE 201.9613884 5856.333333 1211107245 3
RSYNC PUSH RAL STANFORD Yohkoh 139.8397937 9456.75 1271676928 4
RSYNC PUSH RAL UCL CDS 1219.858809 950.3333333 1149635736 3
RSYNC PUSH RAL UCL TRACE 5322.647664 223.5 1211107245 4
RSYNC PUSH RAL UCL Yohkoh 3394.940724 563.6666667 1271676928 3
RSYNC PUSH STANFORD MSSL CDS 48.52608075 23141.33333 1149635757 3
RSYNC PUSH STANFORD MSSL TRACE 498.286099 2373.58 1211107245 1
RSYNC PUSH STANFORD MSSL Yohkoh        
RSYNC PUSH STANFORD UCL CDS 41.36666061 27140 1149635757 1
RSYNC PUSH STANFORD UCL TRACE 374.7855231 3155.73 1211107245 1
RSYNC PUSH STANFORD UCL Yohkoh        
RSYNC PUSH STANFORD RAL CDS 45.06655027 24912 1149635757 2
RSYNC PUSH STANFORD RAL TRACE        
RSYNC PUSH STANFORD RAL Yohkoh        
RSYNC PUSH UCL MSSL CDS 483.1439128 2328.5 1149635757 2
RSYNC PUSH UCL MSSL TRACE 1703.424817 694.32 1211107245 1
RSYNC PUSH UCL MSSL Yohkoh 1359.346746 913.58 1271676928 1
RYSNC PUSH UCL RAL CDS 334.5325295 3356 1149635757 1
RYSNC PUSH UCL RAL TRACE 5331.178359 221.85 1211107245 1
RYSNC PUSH UCL RAL Yohkoh        
RSYNC PUSH UCL STANFORD CDS        
RSYNC PUSH UCL STANFORD TRACE 392.0036057 3017.12 1211107245 1
RSYNC PUSH UCL STANFORD Yohkoh        
RSYNC PUSH MSSL RAL CDS 258.9123048 4399.565 1149635757 2
RSYNC PUSH MSSL RAL TRACE 1864.27901 659.47 1211107245 5
RSYNC PUSH MSSL RAL Yohkoh        
RSYNC PUSH MSSL STANFORD CDS        
RSYNC PUSH MSSL STANFORD TRACE 366.5222661 3443.943333 1211107245 6
RSYNC PUSH MSSL STANFORD Yohkoh        
RSYNC PUSH MSSL UCL CDS        
RSYNC PUSH MSSL UCL TRACE 3208.935466 371.088 1211107245 5
RSYNC PUSH MSSL UCL Yohkoh        
SCP PULL MSSL RAL CDS        
SCP PULL MSSL RAL TRACE        
SCP PULL MSSL RAL Yohkoh        
SCP PULL MSSL STANFORD CDS        
SCP PULL MSSL STANFORD TRACE        
SCP PULL MSSL STANFORD Yohkoh        
SCP PULL MSSL UCL CDS        
SCP PULL MSSL UCL TRACE        
SCP PULL MSSL UCL Yohkoh        
SCP PULL RAL MSSL CDS 550.9280575 2826.6 1149435740 5
SCP PULL RAL MSSL TRACE        
SCP PULL RAL MSSL Yohkoh 2009.796939 1108 1271676928 4
SCP PULL RAL STANFORD CDS 84.18699449 14008.33333 1149635736 3
SCP PULL RAL STANFORD TRACE        
SCP PULL RAL STANFORD Yohkoh 266.6911728 6820.25 1271676928 4
SCP PULL RAL UCL CDS 668.0376442 1794.8 1149635736 5
SCP PULL RAL UCL TRACE        
SCP PULL RAL UCL Yohkoh 2873.355229 462.5 1271676928 4
SCP PULL STANFORD MSSL cds        
SCP PULL STANFORD MSSL TRACE 384.8102238 3073.52 1211107245 1
SCP PULL STANFORD MSSL Yohkoh        
SCP PULL STANFORD RAL cds        
SCP PULL STANFORD RAL TRACE 191.8961779 6186.5 1211107245 2
SCP PULL STANFORD RAL Yohkoh        
SCP PULL STANFORD UCL cds        
SCP PULL STANFORD UCL TRACE 291.0931916 4471.87 1211107245 3
SCP PULL STANFORD UCL Yohkoh        
SCP PULL UCL MSSL CDS 70.3512571 15010 1081315706 1
SCP PULL UCL MSSL TRACE        
SCP PULL UCL MSSL Yohkoh        
SCP PULL UCL RAL cds        
SCP PULL UCL RAL TRACE        
SCP PULL UCL RAL Yohkoh        
SCP PULL UCL STANFORD cds        
SCP PULL UCL STANFORD TRACE        
SCP PULL UCL STANFORD Yohkoh        
SCP PUSH MSSL RAL CDS 1514.116472 4771.123333 1149635757 3
SCP PUSH MSSL RAL TRACE 1936.348918 610.8 1211107245 1
SCP PUSH MSSL RAL Yohkoh        
SCP PUSH MSSL STANFORD CDS 43.50101506 25808.39 1149635750 1
SCP PUSH MSSL STANFORD TRACE 3016.4 3016.4 1211107245 2
SCP PUSH MSSL STANFORD Yohkoh        
SCP PUSH MSSL UCL CDS 231.5858438 24163.805 1152429578 1
SCP PUSH MSSL UCL TRACE        
SCP PUSH MSSL UCL Yohkoh        
SCP PUSH RAL MSSL CDS 462.8397723 2993.6 1149635744 5
SCP PUSH RAL MSSL TRACE        
SCP PUSH RAL MSSL Yohkoh 1879.278741 698.25 1271676928 4
SCP PUSH RAL STANFORD CDS 83.86395588 14217.6 1149635744 5
SCP PUSH RAL STANFORD TRACE        
SCP PUSH RAL STANFORD Yohkoh 102.090398 12388 1271676928 4
SCP PUSH RAL UCL CDS 580.3874277 2054.333333 1149635736 3
SCP PUSH RAL UCL TRACE        
SCP PUSH RAL UCL Yohkoh 3678.889365 354.75 1271676928 4
SCP PUSH STANFORD MSSL CDS 47.34337475 23931.5 1154648019 2
SCP PUSH STANFORD MSSL TRACE 271.5666778 4447 1211107245 3
SCP PUSH STANFORD MSSL Yohkoh        
SCP PUSH STANFORD RAL CDS 45.3153481 24810.5 1149635747 2
SCP PUSH STANFORD RAL TRACE 177.5547844 6747 1211107245 3
SCP PUSH STANFORD RAL Yohkoh        
SCP PUSH STANFORD UCL CDS        
SCP PUSH STANFORD UCL TRACE 278.2749051 4827.15 1211107245 3
SCP PUSH STANFORD UCL Yohkoh        
SCP PUSH UCL MSSL CDS        
SCP PUSH UCL MSSL TRACE        
SCP PUSH UCL MSSL Yohkoh        
SCP PUSH UCL RAL CDS 319.5000338 3513.9 1149635757 2
SCP PUSH UCL RAL TRACE        
SCP PUSH UCL RAL Yohkoh        
SCP PUSH UCL STANFORD CDS 54.34879542 20657.15 1149635754 1
SCP PUSH UCL STANFORD TRACE        
SCP PUSH UCL STANFORD Yohkoh        
SFTP PULL MSSL RAL CDS        
SFTP PULL MSSL RAL TRACE        
SFTP PULL MSSL RAL Yohkoh        
SFTP PULL MSSL STANFORD CDS        
SFTP PULL MSSL STANFORD TRACE        
SFTP PULL MSSL STANFORD Yohkoh        
SFTP PULL MSSL UCL CDS        
SFTP PULL MSSL UCL TRACE        
SFTP PULL MSSL UCL Yohkoh        
SFTP PULL RAL MSSL CDS 1145.359972 984.3333333 1149635736 6
SFTP PULL RAL MSSL TRACE 2431.7182 490.2 1211107245 5
SFTP PULL RAL MSSL Yohkoh 1677.521889 759.6666667 1271676928 3
SFTP PULL RAL STANFORD CDS 79.77291835 14108.16667 1149635736 6
SFTP PULL RAL STANFORD TRACE 192.5336876 6186.714286 1211107245 7
SFTP PULL RAL STANFORD Yohkoh 138.6082475 8985.6 1271676928 5
SFTP PULL RAL UCL CDS 1744.604418 644.75 1149635736 4
SFTP PULL RAL UCL TRACE 3737.637272 318.3333333 1211107245 3
SFTP PULL RAL UCL Yohkoh 2416.093385 514 1271676928 1
SFTP PULL STANFORD MSSL CDS        
SFTP PULL STANFORD MSSL TRACE        
SFTP PULL STANFORD MSSL Yohkoh        
SFTP PULL STANFORD RAL CDS        
SFTP PULL STANFORD RAL TRACE        
SFTP PULL STANFORD RAL Yohkoh        
SFTP PULL STANFORD UCL CDS        
SFTP PULL STANFORD UCL TRACE        
SFTP PULL STANFORD UCL Yohkoh        
SFTP PULL UCL MSSL CDS        
SFTP PULL UCL MSSL TRACE 2078.524338 569.02 1211107245 1
SFTP PULL UCL MSSL Yohkoh        
SFTP PULL UCL RAL CDS        
SFTP PULL UCL RAL TRACE 5783.198469 204.51 1211107245 1
SFTP PULL UCL RAL Yohkoh        
SFTP PULL UCL STANFORD CDS        
SFTP PULL UCL STANFORD TRACE        
SFTP PULL UCL STANFORD Yohkoh        
SFTP PUSH MSSL RAL CDS        
SFTP PUSH MSSL RAL TRACE 1674.25054 725.5314286 1211107245 7
SFTP PUSH MSSL RAL Yohkoh        
SFTP PUSH MSSL STANFORD CDS        
SFTP PUSH MSSL STANFORD TRACE 232.9840935 5078.25 1211107245 4
SFTP PUSH MSSL STANFORD Yohkoh        
SFTP PUSH MSSL UCL CDS        
SFTP PUSH MSSL UCL TRACE 2885.151698 411.8 1211107245 2
SFTP PUSH MSSL UCL Yohkoh        
SFTP PUSH RAL MSSL CDS 3782.20674 801.3333333 1149635736 6
SFTP PUSH RAL MSSL TRACE 2642.108355 456.4 1211107245 5
SFTP PUSH RAL MSSL Yohkoh 1773.040648 702.4 1271676928 5
SFTP PUSH RAL STANFORD CDS 1692.597933 3855.25 1149635736 8
SFTP PUSH RAL STANFORD TRACE 559.6286682 5137.571429 1211107245 7
SFTP PUSH RAL STANFORD Yohkoh 1070.325022 3999.357143 1271676928 14
SFTP PUSH RAL UCL CDS 1613.334598 696.6666667 1149635757 3
SFTP PUSH RAL UCL TRACE 4927.278178 261 1211107245 4
SFTP PUSH RAL UCL Yohkoh 2444.667814 508 1271676928 2
SFTP PUSH STANFORD MSSL CDS        
SFTP PUSH STANFORD MSSL TRACE 228.3963692 5178.5 1211107245 4
SFTP PUSH STANFORD MSSL Yohkoh        
SFTP PUSH STANFORD RAL CDS 141.3325448 8065.5 1149635757 6
SFTP PUSH STANFORD RAL TRACE 197.0703629 6004 1211107245 4
SFTP PUSH STANFORD RAL Yohkoh 152.6766658 8134 1271676928 1
SFTP PUSH STANFORD UCL CDS        
SFTP PUSH STANFORD UCL TRACE 215.378449 5499.5 1211107245 3
SFTP PUSH STANFORD UCL Yohkoh        
SFTP PUSH UCL MSSL CDS        
SFTP PUSH UCL MSSL TRACE 1835.682408 652.87 1211107245 2
SFTP PUSH UCL MSSL Yohkoh        
SFTP PUSH UCL RAL CDS        
SFTP PUSH UCL RAL TRACE 5412.010797 220.5325 1211107245 4
SFTP PUSH UCL RAL Yohkoh        
SFTP PUSH UCL STANFORD CDS        
SFTP PUSH UCL STANFORD TRACE 239.9500845 4935 1211107245 4
SFTP PUSH UCL STANFORD Yohkoh        

Charts

* rsync_pull.jpg:
rsync_pull.jpg

* rsync_push.jpg:
rsync_push.jpg

* scp_pull.jpg:
scp_pull.jpg

* scp_push.jpg:
scp_push.jpg

* sftp_pull.jpg:
sftp_pull.jpg

* sftp_push.jpg:
sftp_push.jpg

Conclusions

Transfer trends

The three protocols tested - rsync, scp and sftp - were all operated non-interactively over ssh, so result times for like transfers (same node pair, same instrument) we expected to be similar for all three protocols. Times were also expected to be similar for transfers between node pairs located on the UK SUPERJANET backbone (MSSL, RAL and UXL). Times were expected to be longer for node pairs involving Stanford as data was sent over transatlantic data lines.

It is disappointing that many tests did not return usable datasets after a week of testing. The most common cause of lapsed tests was socket timeouts; this problem particularly affected transfers of the single 1.14 GB TRACE movie between Stanford and UK nodes. In addition, the MSSL firewall was reset once, and the UCL network (which also affects MSSL) experienced extremely sluggish traffic towards the end of the testing period. Rsync was turned off for the UCL node at midnight on Thursday 20 April 2006. Finally, each test script for a given protocol initiated at a given node was designed to test transfers of all three instrument data types. Therefore, if the script was aborted for any reason, tests at the end of the script were not run. Transfers involving Yohkoh were most susceptible to this risk as the instruments were tested alphabetically. This was corrected during the test week (where noticed) by transferring the Yohkoh tests to the beginning of scripts.

The available datasets exhibit the following trends:

  • Transfers involving Stanford take longer than those between UK nodes. Data transfers from the UK to Stanford take approximately 10 times as long as transfers between UK nodes.
  • Single large files (TRACE movie, 1.14 GB) can be transferred more quickly than the equivalent size of many individual files:
    • CDS - ~ 1.2 GB, 5000 files, average 200 kB each. Longest total transfer, most successful individual file transfers
    • TRACE - 1.14 GB, 1 file. Shortest total transfer, least successful individual file transfers. Twice as fast as CDS transfer for any given protocol / node pair.
    • Yohkoh - 1.2 GB, ~1500 files, ~ 1MB each. 1.5 times as fast as CDS transfer for any given protocol.
  • Fastest transfers were between UCL and RAL. (MSSL connection is over slower microwave link to UCL, Stanford reached via transatlantic link)
  • Transfer times for any given transfer are roughly equivalent for the three protocols. However, transfer times are slightly faster when data is pushed to rather than pulled from a remote machine.
  • The scripts were most difficult to write for the dec alpha operating system on the RAL machine (other machines were linux); however, this machine exhibited the fewest timeouts and network errors, therefore producing the most successful runs per test.

Future Work

Further transfer tests using gridFTP will be undertaken between MSSL and Stanford only. Note: the collected datasets will be added to this wiki page on Monday, 22 May 2006.

Thanks

Many thanks to the system administrators who permitted the network tests on their systems: Rick Bogart (Stanford), Peter Chiu (RAL), John Deacon (UCL) and Paul Lamb (MSSL).

-- ElizabethAuden - 10 May 2006

Topic attachments
I Attachment History Action Size Date Who Comment
Microsoft Excel Spreadsheetxls esdo_mssl_timelog_rsync.xls r1 manage 2205.0 K 2006-05-19 - 14:57 ElizabethAuden  
JPEGjpg rsync_pull.jpg r1 manage 98.1 K 2006-05-19 - 14:51 ElizabethAuden  
JPEGjpg rsync_push.jpg r1 manage 97.0 K 2006-05-19 - 14:52 ElizabethAuden  
JPEGjpg scp_pull.jpg r1 manage 89.6 K 2006-05-19 - 14:52 ElizabethAuden  
JPEGjpg scp_push.jpg r1 manage 89.3 K 2006-05-19 - 14:53 ElizabethAuden  
JPEGjpg sftp_pull.jpg r1 manage 91.6 K 2006-05-19 - 14:54 ElizabethAuden  
JPEGjpg sftp_push.jpg r1 manage 99.7 K 2006-05-19 - 14:55 ElizabethAuden  
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | More topic actions
Topic revision: r8 - 2008-01-06 - ElizabethAuden
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback