70010 Deep Learning
Overview:
Note that this course will be held online hybrid as a combination of pre-recorded lectures, weekly Q&A sessions, tutorials on Teams and individual coding projects.
Q&As and tutorials have been timetabled for 2 hours per week:
In Feb/Mar: Tuesday 14-15 or Huxley 311
In Feb/Mar Tuesday 15-16 or Huxley 311 Tutorial
However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found on Materials and CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes.
Questions can be discussed on the course’s Edstem page (70010/460 Deep Learning [Spring 2022])
Timetable
| Week 1 (starting 9th January) | ||||
| No lectures, no tutorials. | ||||
| Week 2 (starting 16th January) | ||||
| – | pre-recorded | 01 Logistics 02 The curse of dimensionality 03 Convolutions 04 Convolutional Neural Networks | Lecture | Kainz |
| papers to read and notes | N00 Admin Notes N01 Convolution Notes N01a Convolution slides | |||
| Friday 9-10 | HXL 308 (planned) | flipped classroom, panopto recording | Q&A | Kainz |
| Friday 10-11 | MS Teams Lab queue /HXL 308 | MS Teams T01 signals T02 padding and strides | Tutorial | Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li |
| coursework prep. | Introduction to PyTorch I jupyter notebook I Introduction to PyTorch II jupyter notebook II | practical | Pace | |
| Week 3 (starting 23rd January) | ||||
| – | pre-recorded | 05 Equivariance and Invariance 06 LeNet 07 AlexNet 08 VGG | Lecture | Kainz |
| papers to read and notes | LeNet AlexNet VGG N02 Equivariance and Invariance N03 LeNet N04 AlexNet N05 VGG | |||
| Friday 9-10 | MS Teams | MS Teams recording | Q&A | Kainz |
| Friday 10-11 | MS Teams Lab queue | MS Teams T03 CNNs | Tutorial | Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li |
| deadline: TBA, 19:00 | Coursework Task 1 paperspace usage | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here | |||
| Week 4 (starting 30th January) | ||||
| – | pre-recorded | 09 Network in Network and Inception 10 BatchNorm 11 ResNet, DenseNet and beyond 12 Activation functions 13 Loss functions 14 Data Augmentation and case study | Lecture | Kainz |
| papers to read | Inception ResNet BatchNorm N06 Inception N07 BatchNorm N08 ResNet N09 Activation functions N10 Loss functions N11 Augmentation | |||
| Friday 9-10 | MS Teams Lab queue/HXL 308 (planned) | MS Teams | Q&A | Kainz, Vlontzos |
| Friday 10-11 | MS Teams Lab queue | MS Teams T04 Covariate shift T05 Batch-norm | Tutorial | Vlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li |
| deadline: TBA, 19:00 | Coursework Task 1 | assessed | please submit this via LabTS | |
| Week 5 (starting 06th February) | ||||
| Tuesday 14-15 | pre-recorded /HXL 311 | 15 Generative models 16 VAEs 17 GANs 17a GANs advanced | Lecture | Li |
| papers to read | VAEs GANs N15 intro slides N16 VAEs slides N17 GANs slides N17a slides | |||
| Tuesday 14-15 | MS Teams Lab queue/HXL 311 | MS Teams recording | Q&A | Li |
| Tuesday 15-16 | MS Teams Lab queue/HXL 311 | MS Teams T06 VAEs, GANs | Tutorial | Li, Coppock, Spies |
| deadline:TBA, 19:00 | Coursework Task 2 | assessed | Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li | |
| Quiz | test your knowlege here | |||
| Week 6 (starting 15th February) | ||||
| Tuesday 14-15 | pre-recorded /HXL 311 | 18 RNN basics 19 RNN applications 20 Attention & Transformer basics 21 Transformer applications & advanced | Lecture | Li |
| papers to read | N18 RNNs slides N19 slides N20 Transformer & Attention slides N21 slides | |||
| Tuesday 14-15 | MS Teams Lab queue/HXL 311 | MS Teams recording | Q&A | Li, Coppock, Spies |
| Tuesday 15-16 | MS Teams Lab queue/HXL 311 | MS Teams T07 attention T08 recurrent networks | Tutorial | Coppock, Spies, Barmpas, Reynaud, Baugh, Srinivasan, Ma, Tuli, Li |
| deadline: 25 Feb 2021, 19:00 | Coursework Task 2 | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here about RNNs and also here about transformers | |||
| Week 7 (starting 21st February) | ||||
| Friday 9-11 | MS Teams live | 22 GNNs, GCNs | Lecture | Bronstein, Li |
| papers to read | geometric deep learning | |||
| MS Teams Lab queue | MS Teams T09 GCNs | Vlontzos, Coppock, Spies, Barmpas, Reynaud, Baugh, Ma, Tuli, Li | ||
| deadline: 11 Mar 2021, 19:00 | Coursework Task 3 | assessed | team | |
| Quiz | test your knowlege here | |||
| Week 8 (starting 01st March) | ||||
| papers to read | ||||
| Tuesday 14-15 | MS Teams/HXL 311 | annual live panel discussion about the hype MS Teams | Q&A | Li |
| Tuesday 15-16 | MS Teams Lab queue | MS Teams Q&A | Tutorial | Li & team |
| deadline: 11 Mar 2021, 19:00 | Coursework Task 3 | assessed | team | |
| Week 9 (starting 8th March) | ||||
| – | – | no lecture or tutorial | – | – |
Exam
Examinable material from lectures 01-14 is highlighted here with exclamation marks. Note, lectures 15-21 are also examinable but there are no exclamationa marks! The exam will count towards 50% of your final mark.
Coursework
There will be three practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.
We recommend using Google CoLab Paperspace.com with GPU support for testing.
The tasks are embedded into Jupyter notebooks, which also contain the task description.