Skip to content

Introduction to the workshop

Learning outcomes

  • Can find an overview of the course
  • Can find the schedule
  • Have seen the ways to connect at this course
  • Have seen where to put anonymous feedback during the day
  • Have included HPC cluster name in Zoom username, e.g. [Dardel] Sven Svensson
  • Understand when to use which of the file transfer tools discussed today
For teachers

Teaching goals are:

  • Learners can find an overview of the course
  • Learners can find the schedule
  • Learners have seen the ways to connect at this course
  • Learners have seen where to put anonymous feedback during the day
  • Learners have included their HPC cluster name in Zoom username
  • Learners have practiced to deduced from a use case when to use which of the file transfer tools discussed today

Prior questions:

  • What is a file transfer?
  • What does one need to do, to be able to do a file transfer?
  • How does it look like to do a file transfer?
  • Can you name some programs that can do file transfer?

The sessions in this course

As per prerequisites:

  • you already have an account at an HPC cluster
  • you can already log in your HPC cluster

Below an overview of the sessions and their relations.

flowchart TD

  cluster_to_cluster[Cluster to cluster]

  scp --> rsync
  rsync --> sftp
  sftp --> cluster_to_cluster

Tool comparison

Here is an overview of the file transfer tools in this course:

Feature scp rsync sftp
Summary Preinstalled Powerful Two-way portal
Type of user interface Terminal Terminal Terminal
Can transfer files between clusters? Yes Yes Yes
Installation needed? Maybe Yes Maybe
Suitable for large files? Not really Yes Not really
Behavior upon overwriting files Overwrite Overwrite if newer Asks
Behavior upon downloading again Download again Only download new(er) files Download again
Suitable for sensitive data? Yes Yes Yes
Tolerant to interruption? No Yes No
Flexibility Some High Low
Session Session Session Session

Exercises

Exercise 1: rename yourself

Please change your Zoom name to start with your HPC cluster, as [name_of_hpc_cluster] your_name, e.g. [Dardel] Sven Svensson.

Why the name of the HPC cluster first?

It will make Zoom sort your names in a way most convenient for the course, which helps the logistics of the course.

Exercise 2: be assigned a Zoom room

We will use breakout rooms, as this is best for interaction and learning. However, some prefer to work alone. In this course, we allow for both.

For a quick process, we follow the following procedure:

flowchart TD

  prefer_work_together(Prefer to work together?)
  indicate_yes[Use the yes symbol]
  indicate_no[Use the no symbol]

  prefer_work_together --> |Yes| indicate_yes
  prefer_work_together --> |No| indicate_no

You will be ruthlessly moved to a breakout room when possible.

Good luck and enjoy!

Exercise 3: where is what?

Explore the website of the course. Where you can you find the things below?

Exercise 4: which tool to use?

Different tools shine in different uses cases. Which use case match which tool? Here we go through some use cases to conclude the best tool to use. Use the 'Tool comparison' section to answer these questions.

Your colleague is not tech-savvy, does not dare to use the command-line and wants to use 'something easy'. Which file transfer tool would you recommend?

Answer

FileZilla, a graphical tool, would be most suitable here.

This tool is taught in the previous course, 'Connect and File Transfer'.

However, when you need to pick the tools discussed today, use scp, as it is (always?) installed with any terminal.

Another colleague is not tech-savvy, but is fine with using the command-line and wants to use 'something easy'. Which file transfer tool would you recommend?

Answer

scp, as it is (always?) installed with any terminal.

This is the also reason why we teach this first: it allows our learners to see how a successful file transfer looks like.

You are working on a project, where you need to upload and download often. Normally, you'd use FileZilla, as it shows both your local and remote files. However, you could not install FileZilla on your (work) computer. You are happy to use the command line.

Answer

sftp, as it allows easy access to both local and remote files.

You could informally call sftp the terminal equivalent of FileZilla.

Your internet connection is bad. Still, you need to upload and download big (and compressed) files.

Answer

rsync, as it allows interrupted downloads.

You are working together with a colleague on an HPC project. You dream of writing a script that downloads the files that are new/changed in your HPC project folder.

Answer

rsync, as it will only download files if these are new/newer. Technically, this is expressed as 'it only downloads the delta', where 'delta' means the changes to be applied.

Exercise 5: Login!

Not all HPC clusters can do all ways to login.

Don't you remember how to do?

Log in session from "Connect and Transfer" workshop

Done?

Then you are ahead and encouraged to read up on the next page.