70010 Deep Learning
Overview:
Note that this course will be held online as a combination of pre-recorded lectures, weekly Q&A sessions, tutorials on Teams and individual coding projects
Q&As and tutorials have been timetabled for 2 hours per week:
TBA
TBA
However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found on CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes.
Questions can be discussed on the course’s Piazza page (70010/460 Deep Learning [Spring 2021])
Timetable
| Week 1 (starting 6th January) | ||||
| No lectures, no tutorials. | ||||
| Week 2 (starting 11th January) | ||||
| – | pre-recorded | 01 admin [WIP] 02 The curse of dimensionality 03 Convolutions 04 Convolutional Neural Networks | Lecture | Kainz |
| papers to read | – | |||
| date TBA | time TBA | MS Teams | Q&A | Kainz |
| date TBA | time TBA | MS Teams pytorch and notebooks | Tutorial | Kainz, Vlontzos |
| deadline: 22 Jan 2021, 23:59 | Coursework Task 1 | assessed | Kainz, Vlontzos | |
| Week 3 (starting 18th January) | ||||
| – | pre-recorded | 05 Equivariance and Invariance 06 LeNet-5 07 AlexNet 08 VGG | Lecture | Kainz |
| papers to read | LeNet AlexNet VGG | |||
| date TBA | time TBA | MS Teams | Q&A | Kainz |
| date TBA | time TBA | MS Teams CNNs | Tutorial | Kainz, Vlontzos |
| deadline: 22 Jan 2021, 23:59 | Coursework Task 1 | assessed | Kainz, Vlontzos | |
| Week 4 (starting 25th January) | ||||
| – | pre-recorded | 09 Network in Network and Inception 10 BatchNorm 11 ResNet, DenseNet and beyond 12 Loss functions, Activation functions and Data Augmentation | Lecture | Kainz |
| papers to read | Inception ResNet BatchNorm | |||
| date TBA | time TBA | MS Teams | Q&A | Kainz |
| date TBA | time TBA | MS Teams CNN case studies: , | Tutorial | Kainz, Vlontzos |
| deadline: 05 Feb 2021, 23:59 | Coursework Task 2 | assessed | Kainz, Vlontzos, Stacey | |
| Week 5 (starting 1st February) | ||||
| – | pre-recorded | 13 Generative models 14 VAEs 15 GANs | Lecture | Li |
| papers to read | VAEs GANs | |||
| date TBA | time TBA | MS Teams | Q&A | Li |
| date TBA | time TBA | MS Teams generative models | Tutorial | Li, Stacey |
| deadline: 05 Feb 2021, 23:59 | Coursework Task 2 | assessed | Kainz, Vlontzos, Stacey | |
| Week 6 (starting 8th February) | ||||
| – | pre-recorded | 16 RNNs 17 LSTMs 18 GRUs 19 Transformers and attention | Lecture | Li |
| papers to read | ||||
| date TBA | time TBA | MS Teams | Q&A | Li |
| date TBA | time TBA | MS Teams recurrent networks | Tutorial | Li, Stacey |
| deadline: 19 Feb 2021, 23:59 | Coursework Task 3 | assessed | Li, Stacey | |
| Week 7 (starting 15th February) | ||||
| – | pre-recorded | 20 GNNs, GCNs 21 self-supervised/contrastive learning | Lecture | Li, Vlontzos |
| papers to read | ||||
| date TBA | time TBA | MS Teams | Q&A | Li |
| date TBA | time TBA | MS Teams GCNs, RL | Tutorial | Li, Stacey |
| deadline: 19 Feb 2021, 23:59 | Coursework Task 3 | assessed | Li, Stacey | |
| Week 8 (starting 22th February) | ||||
| – | pre-recorded | 21 Supervised vs unsupervised learning, generalisation, overfitting 22 Energy efficiency of deep learning 24 case studies | Lecture | Li, Kainz, Vlontzos, Stacey, Kainz, guests |
| papers to read | ||||
| date TBA | time TBA | 25 live panel discussion about the hype MS Teams | Q&A | Li, Kainz |
| date TBA | time TBA | MS Teams revision | Tutorial | Li, Stacey |
| Week 9 (starting 4th March) | ||||
| – | TBA | no lecture or tutorial | – | – |
| TBA | Coursework Task 3 | assessed | Li, Stacey |
Coursework
There will be three practical coursework tasks; all of them are assessed. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.
We recommend using Google CoLab with GPU support for testing.
The tasks are embedded into Jupyter notebooks, which also contain the task description.