70010 Deep Learning

70010 Deep Learning
Overview:

Note that this course will be held online hybrid as a combination of pre-recorded lectures, weekly recap and Q&A sessions, tutorials on Teams and individual coding projects.

Q&As and tutorials have been timetabled for 2 hours per week:

Fridays 09-10 Huxley 308 and MS Teams
Fridays 10-11 Huxley 308 and MS Teams Lab queue

However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found below, on Scientia and CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes. Coursework submission will be done via Scientia/LabTS. General questions can be discussed on EdStem.

Timetable

Week 1 (starting 9th January)   
     
  No lectures, no tutorials.  
     
Week 2 (starting 16th January)   
308 + online
Friday 9-10
pre-recorded + flipped classroom + MS Teams01 Logistics
02 The curse of dimensionality (YouTube)
03 Convolutions (YouTube)
04 Convolutional Neural Networks (YouTube)
LectureKainz
papers to read and notesN00 Admin Notes
N01 Convolution Notes
N01a Convolution slides
Friday 10-11MS Teams Lab queue /HXL 308MS Teams
T01 signals
T02 padding and strides
TutorialMeng & team
coursework prep., coursework 1
deadline: 03 Feb, 19:00
 Introduction to PyTorch I
jupyter notebook I (YouTube)
Introduction to PyTorch II
jupyter notebook II (YouTube)
practicalPace
   
Week 3 (starting 23rd January)   
308 + online
Friday 9-10
pre-recorded + flipped classroom + MS Teams05 Equivariance and Invariance (YouTube)
06 LeNet (YouTube)
07 AlexNet (YouTube)
08 VGG (YouTube)
LectureKainz
papers to read and notesLeNet
AlexNet
VGG
N02 Equivariance and Invariance
N03 LeNet
N04 AlexNet
N05 VGG
Friday 10-11 MS Teams Lab queue MS Teams
T03 CNNs
TutorialMeng & team
deadline: 03 Feb, 19:00 Coursework Task 1
paperspace usage
assessedplease submit this via LabTS
Quiz test your knowlege here    
Week 4 (starting 30th January)   
308 + online
Friday 9-10
pre-recorded + flipped classroom + + MS Teams09 Network in Network and Inception (YouTube)
10 BatchNorm (YouTube)
11 ResNet, DenseNet and beyond (YouTube)
12 Activation functions (YouTube)
13 Loss functions (YouTube)
14a The U-Net architecture (YouTube)
14 Data Augmentation (YouTube) and case study (YouTube)
LectureKainz
papers to readInception
ResNet
BatchNorm
N06 Inception
N07 BatchNorm
N08 ResNet
N09 Activation functions
N10 Loss functions
N11 Augmentation
Friday 10-11 MS Teams Lab queue MS Teams
T04 Covariate shift
T05 Batch-norm
TutorialMeng & team
deadline: 03 Feb, 19:00 Coursework Task 1assessedplease submit this via LabTS
     
Week 5 (starting 06th February)   
308 + online
Friday 9-10
pre-recorded + flipped classroom + + MS Teams15 Generative models
16 VAEs
17 GANs
17a GANs advanced
LectureLi
papers to readVAEs
GANs
N15 intro slides
N16 VAEs slides
N17 GANs slides
N17a slides
Friday 10-11MS Teams Lab queue/HXL 311MS Teams
T06 VAEs, GANs
TutorialCoppock & team
deadline: 24 Feb 2023, 19:00 Coursework Task 2assessedCoppock & team
 Quiz test your knowlege here  
Week 6 (starting 13th February)   
308 + online
Friday 9-10
pre-recorded + flipped classroom + MS Teams18 RNN basics
19 RNN applications
20 Attention & Transformer basics
21 Transformer applications & advanced
LectureLi
papers to readN18 RNNs slides
N19 slides
N20 Transformer & Attention slides
N21 slides
308 + online
Tuesday 14-15
MS Teams Lab queue/HXL 311MS Teams
recording
Q&ALi
Friday 10-11MS Teams Lab queue/HXL 311 MS Teams
T07 attention
T08 recurrent networks
TutorialCoppock & team
deadline: 24 Feb 2023, 19:00 Coursework Task 2assessedplease submit this via LabTS
 Quiz  test your knowlege here about RNNs
and also here about transformers
  
Week 7 (starting 20st February)   
online
Friday 9-11
MS Teams live annual live panel discussion about the AI hype 
MS Teams
Li
papers to read
Friday 10-11MS Teams Lab queue Coppock & team
deadline: 24 Feb 2023, 19:00 Coursework Task 2assessedteam
Quiz test your knowlege here  
Week 8 (starting 01st March) 22 GNNs, GCNsLecture  Bronstein, Li
papers to read
Friday 10-11MS Teams/HXL 311 MS Teams
T09 GCNs
geometric deep learning
Q&ACoppock & team
Friday 10-11 MS Teams Lab queue MS Teams
Q&A
Tutorial
 
  –assessed  team
Week 9 (starting 8th March)   
no lecture or tutorial
 

Exam

Examinable material from lectures 01-14 is highlighted here with exclamation marks. Note, lectures 15-21 are also examinable but there are no exclamationa marks! The exam will count towards 50% of your final mark.

Coursework

There will be two practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.

We recommend using Google CoLab Paperspace.com with GPU support for testing.

The tasks are embedded into Jupyter notebooks, which also contain the task description.

Reading

Book: Dive into Deep Learning