70010 Deep Learning

70010 Deep Learning
Overview:

Note that this course will be held online as a combination of pre-recorded lectures, weekly Q&A sessions, tutorials on Teams and individual coding projects

Q&As and tutorials have been timetabled for 2 hours per week:

TBA
TBA

However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found on CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes.

Questions can be discussed on the course’s Piazza page (70010/460 Deep Learning [Spring 2021])

Timetable

Week 1 (starting 6th January)   
     
  No lectures, no tutorials.  
     
Week 2 (starting 11th January)   
pre-recorded01 admin [WIP]
02 The curse of dimensionality
03 Convolutions
04 Convolutional Neural Networks
LectureKainz
papers to read
date TBAtime TBAMS TeamsQ&AKainz
date TBAtime TBAMS Teams
pytorch and notebooks
TutorialKainz, Vlontzos
deadline: 22 Jan 2021, 23:59 Coursework Task 1assessedKainz, Vlontzos 
     
Week 3 (starting 18th January)   
pre-recorded05  Equivariance and Invariance
06 LeNet-5
07 AlexNet
08 VGG
LectureKainz
papers to readLeNet
AlexNet
VGG
date TBAtime TBAMS TeamsQ&AKainz
date TBAtime TBAMS Teams
CNNs
TutorialKainz, Vlontzos
deadline: 22 Jan 2021, 23:59 Coursework Task 1assessedKainz, Vlontzos
     
Week 4 (starting 25th January)   
pre-recorded09 Network in Network and Inception
10 BatchNorm
11 ResNet, DenseNet and beyond
12 Loss functions, Activation functions and Data Augmentation
LectureKainz
papers to readInception
ResNet
BatchNorm
date TBAtime TBAMS TeamsQ&AKainz
date TBAtime TBAMS Teams
CNN case studies: ,
TutorialKainz, Vlontzos
deadline: 05 Feb 2021, 23:59 Coursework Task 2assessedKainz, Vlontzos, Stacey
     
Week 5 (starting 1st February)   
pre-recorded13 Generative models
14 VAEs
15 GANs
LectureLi
papers to readVAEs
GANs
date TBAtime TBAMS TeamsQ&ALi
date TBAtime TBAMS Teams
generative models
TutorialLi, Stacey
deadline: 05 Feb 2021, 23:59 Coursework Task 2assessedKainz, Vlontzos, Stacey
     
Week 6 (starting 8th February)   
pre-recorded16 RNNs
17 LSTMs
18 GRUs
19 Transformers and attention
LectureLi
papers to read
date TBAtime TBAMS TeamsQ&ALi
date TBAtime TBAMS Teams
recurrent networks
TutorialLi, Stacey
deadline: 19 Feb 2021, 23:59 Coursework Task 3assessedLi, Stacey
     
Week 7 (starting 15th February)   
pre-recorded20 GNNs, GCNs
21 self-supervised/contrastive learning
LectureLi, Vlontzos
papers to read
date TBAtime TBAMS TeamsQ&ALi
date TBAtime TBAMS Teams
GCNs, RL
TutorialLi, Stacey
deadline: 19 Feb 2021, 23:59 Coursework Task 3assessedLi, Stacey
     
Week 8 (starting 22th February)   
pre-recorded21 Supervised vs unsupervised learning, generalisation, overfitting
22 Energy efficiency of deep learning
24 case studies
LectureLi, Kainz, Vlontzos, Stacey, Kainz, guests
papers to read
date TBAtime TBA25 live panel discussion about the hype MS TeamsQ&ALi, Kainz
date TBAtime TBAMS Teams
revision
TutorialLi, Stacey
 
     
Week 9 (starting 4th March)   
TBAno lecture or tutorial
TBA Coursework Task 3assessedLi, Stacey

Coursework

There will be three practical coursework tasks; all of them are assessed. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.

We recommend using Google CoLab with GPU support for testing.

The tasks are embedded into Jupyter notebooks, which also contain the task description.

Reading

Book: Dive into Deep Learning