70010 Deep Learning

70010 Deep Learning
Overview:

Note that this course will be held online hybrid as a combination of pre-recorded lectures, weekly Q&A sessions, tutorials on Teams and individual coding projects.

Q&As and tutorials have been timetabled for 2 hours per week:

Tuesday 14-15 or Friday 9-10
Tuesday 15-16 or Friday 10-11 Tutorial

However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found on Materials and CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes.

Questions can be discussed on the course’s Edstem page (70010/460 Deep Learning [Spring 2022])

Timetable

Week 1 (starting 11th January)   
     
  No lectures, no tutorials.  
     
Week 2 (starting 18th January)   
pre-recorded01 Logistics
02 The curse of dimensionality
03 Convolutions
04 Convolutional Neural Networks
LectureKainz
papers to read and notesN00 Admin Notes
N01 Convolution Notes
N01a Convolution slides
Friday 9-10HXL 308 (planned)flipped classroom, panopto
recording
Q&AKainz, Vlontzos
Friday 10-11MS Teams Lab queue /HXL 308MS Teams
T01 signals
T02 padding and strides
TutorialVlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
coursework prep. Introduction to PyTorch I
jupyter notebook I
Introduction to PyTorch II
jupyter notebook II
practicalPace
   
Week 3 (starting 25th January)   
pre-recorded05 Equivariance and Invariance
06 LeNet
07 AlexNet
08 VGG
LectureKainz
papers to read and notesLeNet
AlexNet
VGG
N02 Equivariance and Invariance
N03 LeNet
N04 AlexNet
N05 VGG
Friday 9-10 MS Teams MS Teams
recording
Q&AKainz, Vlontzos
Friday 10-11 MS Teams Lab queue MS Teams
T03 CNNs
TutorialVlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
deadline: 04 Feb 2022, 19:00 Coursework Task 1assessedVlontzos
Quiz test your knowlege here    
Week 4 (starting 01st February)   
pre-recorded09 Network in Network and Inception
10 BatchNorm
11 ResNet, DenseNet and beyond
12 Activation functions
13 Loss functions
14 Data Augmentation and case study
LectureKainz
papers to readInception
ResNet
BatchNorm
N06 Inception
N07 BatchNorm
N08 ResNet
N09 Activation functions
N10 Loss functions
N11 Augmentation
Friday 9-10 MS Teams Lab queue/HXL 308 (planned)MS TeamsQ&AKainz, Vlontzos
Friday 10-11 MS Teams Lab queue MS Teams
T04 Covariate shift
T05 Batch-norm
TutorialVlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
deadline: 04 Feb 2021, 19:00 Coursework Task 1assessedVlontzos & team
     
Week 5 (starting 08th February)   
Tuesday 14-15/
Friday 9-10 ?
pre-recorded /HXL 308 ?15 Generative models
16 VAEs
17 GANs
17a GANs advanced
LectureLi
papers to readVAEs
GANs
N15 intro slides
N16 VAEs slides
N17 GANs slides
N17a slides
Tuesday 14-15/
Friday 9-10 ?
MS Teams Lab queue/HXL 308 MS Teams
recording
Q&ALi
Tuesday 15-16/
Friday 10-11?
MS Teams Lab queue/HXL 308?MS Teams
T06 VAEs, GANs
TutorialLi, Coppock, Spies
deadline: 25 Feb 2021, 19:00 Coursework Task 2assessedVlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
 Quiz test your knowlege here  
Week 6 (starting 15th February)   
Tuesday 14-15/
Friday 9-10 ?
pre-recorded /HXL 308 ? 18 RNN basics
19 RNN applications
20 Attention & Transformer basics
21 Transformer applications & advanced
LectureLi
papers to readN18 RNNs slides
N19 slides
N20 Transformer & Attention slides
N21 slides
Tuesday 14-15/
Friday 9-10 ?
MS Teams Lab queue/HXL 308? MS Teams
recording
Q&ALi, Coppock, Spies
Tuesday 15-16/
Friday 10-11?
MS Teams Lab queue/HXL 308? MS Teams
T07 attention
T08 recurrent networks
TutorialVlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
deadline: 25 Feb 2021, 19:00 Coursework Task 2assessedCoppock, Spies, & team
     
Week 7 (starting 22th February)   
Tuesday 14-15/
Friday 9-10 ?
MS Teams/HXL 308 ? 22 GNNs, GCNsLectureBronstein, Li
papers to readgeometric deep learning
Tuesday 14-15/
Friday 9-10 ?
MS Teams/HXL 308 ? MS TeamsQ&ABronstein, Li
Tuesday 15-16/
Friday 10-11?
MS Teams Lab queue/HXL 308? MS Teams
T09 GCNs
Vlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li
deadline: 11 Mar 2021, 19:00 Coursework Task 3assessedteam
Quiz test your knowlege here  
Week 8 (starting 01st March)   
papers to read
Tuesday 14-15/
Friday 9-10 ?
MS Teams/HXL 308 ? yearly live panel discussion about the hype MS TeamsQ&ALi
Tuesday 15-16/
Friday 10-11?
MS Teams/HXL 308 ? MS Teams
revision
TutorialLi & team
 
deadline: 11 Mar 2021, 19:00  Coursework Task 3assessed  team
Week 9 (starting 8th March)   
no lecture or tutorial
 

Exam

Examinable material from lectures 01-14 is highlighted here with exclamation marks. Note, lectures 15-21 are also examinable but there are no exclamationa marks! The exam will count towards 50% of your final mark.

Coursework

There will be three practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.

We recommend using Google CoLab Paperspace.com with GPU support for testing.

The tasks are embedded into Jupyter notebooks, which also contain the task description.

Reading

Book: Dive into Deep Learning