Tutorial: Forward propagation

Expected Preparation

Learning Events to be Completed Before

Lecture: Forward propagation

Obligatory for Students

None.

Optional for Students

Matrices multiplication
Getting started with Numpy
Knowledge of linear and logistic regression ([Lecture: Linear Regression]

References and background for students:

John D Kelleher and Brain McNamee. (2018), Fundamentals of Machine Learning for Predictive Data Analytics, MIT Press.
Michael Nielsen. (2015), Neural Networks and Deep Learning, 1. Determination press, San Francisco CA USA.
Charu C. Aggarwal. (2018), Neural Networks and Deep Learning, 1. Springer
Antonio Gulli,Sujit Pal. Deep Learning with Keras, Packt, [ISBN: 9781787128422].

Recommended for Teachers

None.

Instructions for Teachers

Here the goal is to show the implementation steps of the transformer network for images. We also implement a basic CNN with a similar number of trainable parameters, so we can compare the performance of the CNN to the ViT model.

Outline

Duration (min)	Description
15	Training a simple convolutional neural network
35	Training a vision transformer (ViT) neural network
10	Comparing the performance of the two different approaches

Acknowledgements

The Human-Centered AI Masters programme was co-financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.

The materials of this learning event are available under CC BY-NC-SA 4.0

The HCAIM consortium consists of three excellence centres, three SMEs and four Universities

Colofon

Het arrangement Tutorial: Forward propagation is gemaakt met Wikiwijs van Kennisnet. Wikiwijs is hét onderwijsplatform waar je leermiddelen zoekt, maakt en deelt.

Auteur: HCAIM Consortium
Laatst gewijzigd: 2024-05-15 13:51:29
Licentie: Dit lesmateriaal is gepubliceerd onder de Creative Commons Naamsvermelding-GelijkDelen 4.0 Internationale licentie. Dit houdt in dat je onder de voorwaarde van naamsvermelding en publicatie onder dezelfde licentie vrij bent om:

het werk te delen - te kopiëren, te verspreiden en door te geven via elk medium of bestandsformaat

het werk te bewerken - te remixen, te veranderen en afgeleide werken te maken

voor alle doeleinden, inclusief commerciële doeleinden.

Meer informatie over de CC Naamsvermelding-GelijkDelen 4.0 Internationale licentie.