Transfer of files¶
In order to copy data to and from Arrhenius, you can use any SSH or SFTP-based tool (e.g rsync, scp, sftp, FileZilla, ...). Later more file transfer mechanisms might be implemented.
The Arrhenius hostname to use login.hpc.arrhenius.naiss.se.
If you have copied data to or from e.g Tetralith, this should already be familiar to you. If not, there are many rsync guides available online for beginners and pros alike, e.g 1, 2, 3, 4, 5 or 6.
When transferring large amounts of data, we recommend that you use a tool (e.g rsync) that can resume an interrupted transfer.
When moving data within Arrhenius, e.g between two project storage
directories, you can use any normal Linux commands, but you should be
aware that e.g mv is not instantaneous since the project quota
counters needs to be updated which makes the process a little slower
than you might expect.
If you want to make rsync do something other than the "preserve mostly
everything" that the -a option does, read the man
page (man rsync on
the cluster).
Basic example 1: push data from Tetralith to Arrhenius
[x_examp@tetralith1 ~]$ rsync -av /proj/somedirectory/somesubdirectory/ myarrheniususer@login.hpc.arrhenius.naiss.se:/nobackup/proj/disk/somedirectory/somewhere/
(myarrheniususer@login.hpc.arrhenius.naiss.se) Password: <ENTER YOUR ARRHENIUS PASSWORD HERE>
(myarrheniususer@login.hpc.arrhenius.naiss.se) Verification code: <ENTER YOUR ARRHENIUS 2FA CODE HERE>
sending incremental file list
created directory /nobackup/proj/disk/somedirectory/somewhere
[...]
[x_examp@tetralith1 ~]$
Basic example 2: pull data from Tetralith to Arrhenius, preserve hard links and handle sparse files efficiently:
[myarrheniususer@arrhenius11 ~]$ rsync -av -H --sparse x_examp@tetralith:/proj/somedirectory/somesubdirectory/ /nobackup/proj/disk/somedirectory/somewhere/
(x_examp@tetralith.nsc.liu.se) Password: <ENTER YOUR TETRALITH PASSWORD HERE>
(x_examp@tetralith.nsc.liu.se) Verification code: <ENTER YOUR TETRALITH 2FA CODE HERE>
sending incremental file list
created directory /nobackup/proj/disk/somedirectory/somewhere/
[...]
[myarrheniususer@arrhenius11 ~]$
Source repository