70010/97111 Deep Learning
Overview:
Note that this course will be held as a combination of pre-recorded lectures, weekly lectures and recap and Q&A sessions, tutorials in class and on Teams and individual coding projects.
Lectures and tutorials have been timetabled for 2 hours per week:
Fridays 09-10 Huxley 308 and MS Teams
Fridays 10-11 Huxley 308 and MS Teams Lab queue
However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found below, on Scientia. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes. Coursework submission will be done via Scientia/LabTS. General questions can be discussed on EdStem.
Timetable
| Week 1 (starting 13th January) | ||||
| No lectures, no tutorials. | ||||
| Week 2 (starting 20th January) | ||||
| 308 + online Friday 9-10 | pre-recorded + lecture + MS Teams | 01 Logistics 02 Motivation, the curse of dimensionality and basic CNN building blocks (YouTube) watch 02 before the lecture please, we will spend a lot of time on logistics | Lecture | Kainz |
| papers to read and notes | N01 Admin Notes N02 Lecture Notes N02a Lecture slides | |||
| Friday 10-11 | MS Teams Lab queue /HXL 308 | MS Teams/308 T01 Signals T02 Padding and strides | Tutorial | team |
| coursework preparation – coursework 1 deadline: 07 Feb, 19:00 | 1. Introduction to PyTorch I (YouTube) 2. jupyter notebook I 3. Introduction to PyTorch II (YouTube) 4. jupyter notebook II | practical | Pace/GTAs | |
| Week 3 (starting 27th January) | ||||
| 308 + online Friday 9-10 | pre-recorded + lecture + MS Teams | 03 Activation functions and Loss functions (YouTube) | Lecture | Kainz |
| papers to read and notes | N03 Lecture Notes N03a Lecture Slides | |||
| Friday 10-11 | MS Teams Lab queue | MS Teams/308 T03 CNNs | Tutorial | team |
| deadline: 07 Feb, 19:00 | Coursework Task 1 paperspace usage | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here | |||
| Week 4 (starting 03rd February) | ||||
| 308 + online Friday 9-10 | pre-recorded + lecture + MS Teams | 04 Popular Network architectures (and BatchNorm) (YouTube) 05 The U-Net architecture for image segmentation (YouTube) 06 Data Augmentation (YouTube) and case study (YouTube) | Lecture | Kainz |
| papers to read | LeNet AlexNet VGG BatchNorm ResNet Squeeze-and-Excite N04 Lecture Notes N04a Lecture Slides | |||
| Friday 10-11 | MS Teams Lab queue | MS Teams T04 Batch-norm | Tutorial | team |
| deadline: 07 Feb, 19:00 | Coursework Task 1 | assessed | please submit this via LabTS | |
| Week 5 (starting 10th February) | ||||
| 308 + online Friday 9-10 | pre-recorded + lecture + MS Teams | 07 Generative models 08 VAEs 09 GANs 10 Generative Models: Advances | Lecture | Li |
| papers to read | VAEs GANs N07 intro — slides N08 VAEs — slides N09 GANs — slides N10 GANs advances — slides | |||
| Friday 10-11 | MS Teams Lab queue/HXL 311 | MS Teams T05 VAEs, GANs | Tutorial | team |
| deadline: 28 Feb, 19:00 | Coursework Task 2 | assessed | team | |
| Quiz | test your knowlege here | |||
| Week 6 (starting 17th February) | ||||
| 308 + online Friday 9-10 | pre-recorded + lecture + MS Teams | 11 RNN basics 12 RNN applications 13 Attention & Transformer basics 14 Transformer applications & advanced | Lecture | Li |
| papers to read | N11 RNNs — slides N12 RNN applications N13 Transformer & Attention — slides N14 Attention advanced — slides | |||
| 308 + online Tuesday 14-15 | MS Teams Lab queue/HXL 308 | MS Teams recording | Q&A | Li |
| Friday 10-11 | MS Teams Lab queue/HXL 308 | MS Teams T06 recurrent networks | Tutorial | team |
| deadline: 28 Feb, 19:00 | Coursework Task 2 | assessed | please submit this via LabTS | |
| Quiz | test your knowlege here about RNNs and also here about transformers | |||
| Week 7 (starting 24th February) | ||||
| online Friday 9-10 | MS Teams live | 15 Advanced attention and Transformers MS Teams test your knowlege | Coppock | |
| papers to read | ||||
| Friday 10-11 | T07 attention | Coppock | ||
| Friday 10-11 | MS Teams Lab queue | team | ||
| deadline: 28 Feb, 19:00 | Coursework Task 2 | assessed | team | |
| Quiz | ||||
| Week 8 (starting 03rd March) | ||||
| Friday 9-10 | 16 Foundation models: tokenisation, evaluation and scaling | Lecture | Coppock | |
| papers to read | ||||
| Friday 10:00-11 | MS Teams Lab queue | MS Teams Q&A | Lab queue | |
| – | – | assessed | team | |
| Week 9 (starting 10th March) | ||||
| online Friday 9-10 | MS Teams Lab queue | exam revision | – | Li/Coppock/Kainz |
Exam
Examinable material from lectures 01-07 is highlighted with exclamation marks. Note, lectures from the second part and the foundation model part are also examinable in full! The exam will count towards 50% of your final mark.
Coursework
There will be two practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via LabTS. Resulting jupyter notebook files and model weights need to be submitted via LabTS.
We recommend using Google CoLab Paperspace.com the extended DoC GPU cluster with GPU support for develeopment and testing.
The tasks are embedded into Jupyter notebooks, which also contain the task description.