70010/97111 Deep Learning from 2024

70010/97111 Deep Learning
Overview:

Note that this course will be held as a combination of pre-recorded lectures, weekly lectures and recap and Q&A sessions, tutorials in class and on Teams and individual coding projects.

Lectures and tutorials have been timetabled for 2 hours per week:

Fridays 09-10 Huxley 308 and MS Teams
Fridays 10-11 Huxley 308 and MS Teams Lab queue

However, not all timetabled slots will be used every week so please check the timetable below for more information. All notes, tutorials and coursework (including coursework hand-out/in dates) can be found below, on Scientia. Revision notes on essential machine learning can be found here based on Tom Eccles’ original notes. Coursework submission will be done via Scientia/LabTS. General questions can be discussed on EdStem.

Timetable

Week 1 (starting 8th January)   
     
  No lectures, no tutorials.  
     
Week 2 (starting 15th January)   
308 + online
Friday 9-10
pre-recorded + lecture + MS Teams01 Logistics
02 Motivation, the curse of dimensionality and basic CNN building blocks (YouTube)
watch 02 before the lecture please, we will spend a lot of time on logistics
LectureKainz
papers to read and notesN01 Admin Notes
N02 Lecture Notes
N02a Lecture slides
Friday 10-11MS Teams Lab queue /HXL 308MS Teams/308
T01 Signals
T02 Padding and strides
Tutorialteam
coursework preparation – coursework 1
deadline: 03 Feb, 19:00
 1. Introduction to PyTorch I (YouTube)
2. jupyter notebook I
3. Introduction to PyTorch II (YouTube)
4. jupyter notebook II
practicalPace
   
Week 3 (starting 22nd January)   
308 + online
Friday 9-10
pre-recorded + lecture + MS Teams03 Activation functions and Loss functions (YouTube)LectureKainz
papers to read and notesN03 Lecture Notes
N03a Lecture Slides
Friday 10-11 MS Teams Lab queue MS Teams/308
T03 CNNs
Tutorialteam
deadline: 06 Feb, 19:00 Coursework Task 1
paperspace usage
assessedplease submit this via LabTS
Quiz test your knowlege here    
Week 4 (starting 29th January)   
308 + online
Friday 9-10
pre-recorded + lecture + MS Teams04 Popular Network architectures (and BatchNorm) (YouTube)
05 The U-Net architecture for image segmentation (YouTube)
06 Data Augmentation (YouTube) and case study (YouTube)
LectureKainz
papers to readLeNet
AlexNet
VGG
BatchNorm
ResNet
Squeeze-and-Excite
N04 Lecture Notes
N04a Lecture Slides
Friday 10-11 MS Teams Lab queue MS Teams
T04 Batch-norm
Tutorialteam
deadline: 06 Feb, 19:00 Coursework Task 1assessedplease submit this via LabTS
     
Week 5 (starting 05th February)   
308 + online
Friday 9-10
pre-recorded + lecture + MS Teams07 Generative models
08 VAEs
09 GANs
10 Generative Models: Advances
LectureLi
papers to readVAEs
GANs
N07 intro — slides
N08 VAEs — slides
N09 GANs — slides
N10 GANs advances — slides
Friday 10-11MS Teams Lab queue/HXL 311MS Teams
T05 VAEs, GANs
Tutorialteam
deadline: 23 Feb 2023, 19:00 Coursework Task 2assessedteam
 Quiz test your knowlege here  
Week 6 (starting 12th February)   
308 + online
Friday 9-10
pre-recorded + lecture + MS Teams11 RNN basics
12 RNN applications
13 Attention & Transformer basics
14 Transformer applications & advanced
LectureLi
papers to readN11 RNNs — slides
N12 RNN applications
N13 Transformer & Attention — slides
N14 Attention advanced — slides
308 + online
Tuesday 14-15
MS Teams Lab queue/HXL 308MS Teams
recording
Q&ALi
Friday 10-11MS Teams Lab queue/HXL 308 MS Teams
T07 recurrent networks
Tutorialteam
deadline: 23 Feb 2023, 19:00 Coursework Task 2assessedplease submit this via LabTS
 Quiz  test your knowlege here about RNNs
and also here about transformers
  
Week 7 (starting 19th February)   
online
Friday 9-10
MS Teams live 15 Advanced attention and Transformers
MS Teams
Coppock
papers to read
Friday 10-11T06 attentionCoppock
Friday 10-11MS Teams Lab queue team
deadline: 23 Feb 2023, 19:00 Coursework Task 2assessedteam
Quiz test your knowlege here  
Week 8 (starting 26th February)   
Friday 9-1016 Foundation models: tokenisation, evaluation and scaling Lecture Coppock
papers to read

Friday 10:00-11 MS Teams Lab queue MS Teams
Q&A
Lab queue
 
  –assessed  team
Week 9 (starting 4th March)   
online
Friday 9-10
MS Teams Lab queue exam revisionLi/Coppock/Kainz
 

Exam

Examinable material from lectures 01-07 is highlighted with exclamation marks. Note, lectures from the second part are also examinable! The exam will count towards 50% of your final mark.

Coursework

There will be two practical coursework tasks; all of them are assessed. Assessment results count 50% of the final mark. Tasks must be implemented individually and submitted via CATe. Resulting jupyter notebook files and model weights need to be submitted via LabTS.

We recommend using Google CoLab Paperspace.com with GPU support for develeopment and testing.

The tasks are embedded into Jupyter notebooks, which also contain the task description.

Reading

Book: Dive into Deep Learning