70010 Deep Learning
Overview:
Note that this course will be held online hybrid as a combination of pre-recorded lectures, weekly recap and Q&A sessions, tutorials on Teams and individual coding projects.
Q&As and tutorials have been timetabled for 2 hours per week:
Fridays 09-10 Huxley 308 and MS Teams
Fridays 10-11 Huxley 308 and MS Teams Lab queue
However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found below, on Scientia and CATe. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes. Coursework submission will be done via Scientia/LabTS. General questions can be discussed on EdStem.
Timetable
| Week 1 (starting 9th January) | ||||
| No lectures, no tutorials. | ||||
| Week 2 (starting 16th January) | ||||
| 308 + online Friday 9-10 | pre-recorded + flipped classroom + MS Teams | 01 Logistics 02 The curse of dimensionality (YouTube) 03 Convolutions (YouTube) 04 Convolutional Neural Networks (YouTube) | Lecture | Kainz |
| papers to read and notes | N00 Admin Notes N01 Convolution Notes N01a Convolution slides | |||
| Friday 10-11 | MS Teams Lab queue /HXL 308 | MS Teams T01 signals T02 padding and strides | Tutorial | Meng & team |
| coursework prep., coursework 1 deadline: 03 Feb, 19:00 | Introduction to PyTorch I jupyter notebook I (YouTube) Introduction to PyTorch II jupyter notebook II (YouTube) | practical | Pace | |
| Week 3 (starting 23rd January) | ||||
| 308 + online Friday 9-10 | pre-recorded + flipped classroom + MS Teams | 05 Equivariance and Invariance (YouTube) 06 LeNet (YouTube) 07 AlexNet (YouTube) 08 VGG (YouTube) | Lecture | Kainz |
| papers to read and notes | LeNet AlexNet VGG N02 Equivariance and Invariance N03 LeNet N04 AlexNet N05 VGG | |||
| Friday 10-11 | MS Teams Lab queue | MS Teams T03 CNNs | Tutorial | Meng & team |
| deadline: 03 Feb, 19:00 | Coursework Task 1 paperspace usage | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here | |||
| Week 4 (starting 30th January) | ||||
| 308 + online Friday 9-10 | pre-recorded + flipped classroom + + MS Teams | 09 Network in Network and Inception (YouTube) 10 BatchNorm (YouTube) 11 ResNet, DenseNet and beyond (YouTube) 12 Activation functions (YouTube) 13 Loss functions (YouTube) 14a The U-Net architecture (YouTube) 14 Data Augmentation (YouTube) and case study (YouTube) | Lecture | Kainz |
| papers to read | Inception ResNet BatchNorm N06 Inception N07 BatchNorm N08 ResNet N09 Activation functions N10 Loss functions N11 Augmentation | |||
| Friday 10-11 | MS Teams Lab queue | MS Teams T04 Covariate shift T05 Batch-norm | Tutorial | Meng & team |
| deadline: 03 Feb, 19:00 | Coursework Task 1 | assessed | please submit this via LabTS | |
| Week 5 (starting 06th February) | ||||
| 308 + online Friday 9-10 | pre-recorded + flipped classroom + + MS Teams | 15 Generative models 16 VAEs 17 GANs 17a GANs advanced | Lecture | Li |
| papers to read | VAEs GANs N15 intro slides N16 VAEs slides N17 GANs slides 17a N17b slides | |||
| Friday 10-11 | MS Teams Lab queue/HXL 311 | MS Teams T06 VAEs, GANs | Tutorial | Coppock & team |
| deadline: 24 Feb 2023, 19:00 | Coursework Task 2 | assessed | Coppock & team | |
| Quiz | test your knowlege here | |||
| Week 6 (starting 13th February) | ||||
| 308 + online Friday 9-10 | pre-recorded + flipped classroom + MS Teams | 18 RNN basics slides 19 RNN applications 20 Attention & Transformer basics 21 Transformer applications & advanced | Lecture | Li |
| papers to read | N18 RNNs slides N19 RNN applications N20 Transformer & Attention slides N21 Attention advanced slides | |||
| 308 + online Tuesday 14-15 | MS Teams Lab queue/HXL 311 | MS Teams recording | Q&A | Li |
| Friday 10-11 | MS Teams Lab queue/HXL 311 | MS Teams T07 attention T08 recurrent networks | Tutorial | Coppock & team |
| deadline: 24 Feb 2023, 19:00 | Coursework Task 2 | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here about RNNs and also here about transformers | |||
| Week 7 (starting 20st February) | ||||
| online Friday 9-11 | MS Teams live | annual live panel discussion about the AI hype MS Teams | Li | |
| papers to read | ||||
| Friday 10-11 | MS Teams Lab queue | Coppock & team | ||
| deadline: 24 Feb 2023, 19:00 | Coursework Task 2 | assessed | team | |
| Quiz | test your knowlege here | |||
| Week 8 (starting 01st March) | ||||
| Friday 9-10:30 | 22 GNNs, GCNs | Lecture | Bronstein, Li | |
| papers to read | Geometric deep learning: going beyond euclidean data | |||
| Friday 10-11 | MS Teams/HXL 311 | MS Teams geometric deep learning | Q&A | Coppock & team |
| Friday 10-11 | MS Teams Lab queue | MS Teams Q&A | Tutorial | |
| – | – | assessed | team | |
| Week 9 (starting 8th March) | ||||
| – | – | no lecture or tutorial | – | – |
Exam
Examinable material from lectures 01-14 is highlighted here with exclamation marks. Note, lectures 15-21 are also examinable but there are no exclamationa marks! The exam will count towards 50% of your final mark.
Coursework
There will be two practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted in a zip archive.
We recommend using Google CoLab Paperspace.com with GPU support for testing.
The tasks are embedded into Jupyter notebooks, which also contain the task description.