Welcome to the course: AI and HPC¶
Learning outcomes
After this course, you will
- Be familiar with the history and context of deep learning
- Be familiar with some of the principles of deep learning (backpropagation, training concepts)
- Be familiar with some deep learning architectures (perceptron, DNN, CNN, recurrent, transformer)
- Be familiar with PyTorch and master basic constructs
- Be able to run simpler AI tasks
- Be familiar with an instance of a hybrid HPC architecture (CPU+GPU)
- Be familiar with different methods of parallelizing AI tasks
- Master some aspects of running AI tasks on an HPC system
- Be aware of some of the risks and implications of AI for society
Introduction¶
This course is a collaboration between Epicure and NAISS.
The term AI is nowadays often used interchangeably with Deep Neural Networks, a technology which is currently changing society as well as science and technology. There is a strong connection between AI and High Performance Computing (HPC) in that AI workloads often need to be, or are run on, HPC systems. Such workloads also turn out to be a particularly challenging application of HPC.
This hybrid (on location/online) course is at the beginners/intermediate level and is mainly intended for students familiar with HPC and in need of an introduction to Deep Learning and how to run AI workloads on HPC clusters. Some of the basic theory and models in deep learning will be covered as well as the techniques involved when running such models. The course includes practical exercises to deepen the understanding of the lectures. At the end of the course participants should be familiar with some of the concepts in deep learning, be able to run simple AI workloads and run them in parallel on an HPC system. The teaching language will be English.
Time and place¶
The course will be given June 3-5 at:
Salongen Osquars backe 31, Stockholm Room 2207, second floor KTH library KTH Campus
Online participation via Zoom is also available. The Zoom link will be emailed to registered participants.
Registration¶
To register, please fill in the registration form.
Deadline for registration is May 20:th.
Preliminary schedule¶
Day 1: Deep learning¶
| Time | Topic |
|---|---|
| 09:00 | Welcome and course overview |
| 09:10 | History of AI |
| 09:45 | Coffee break |
| 10:00 | Basic theory of neural networks |
| 11:00 | PyTorch |
| 12:00 | Lunch |
| 13:00 | Introduction to Dardel with Containers |
| 14:00 | Intro to hands-on session |
| 14:30 | Hands-on session |
| 16:00 | End of day |
Day 2:¶
| Time | Topic |
|---|---|
| 09:00 | Convolutional neural networks |
| 09:50 | Break |
| 10:00 | Recurrent neural networks |
| 11:00 | Transformer models and LLMs |
| 12:00 | Lunch |
| 13:00 | Training techniques |
| 14:00 | Hands-on session |
| 16:00 | End of day |
Day 3:¶
| Time | Topic |
|---|---|
| 09:00 | Architecture of HPC hardware for AI |
| 09:50 | Break |
| 10:00 | HPC system architecture and AI |
| 11:00 | Methods of parallelization |
| 12:00 | Lunch |
| 13:00 | Societal and philosophical issues with AI (AI safety) |
| 14:00 | Hands-on |
Comments and questions¶
Comments and questions on these training events should be sent to NAISS using the support form in SUPR. Please select “Question about a training event” as “Problem or Questions Type”.