Transformer
| Title | Transformer networks | |
| Duration | 60 min | |
| Module | B | |
| Lesson Type | Lecture | |
| Focus | Technical - Deep Learning | |
| Topic |
Transformer |
sequence-to-sequence learning, seq2seq, attention mechanism, self-attention mechanism, transformer network,
None.
None.
None.
The materials of this learning event are available under CC BY-NC-SA 4.0.
In the lecture first we just briefly repeat what we learned about sequential data previously (e.g. in the RNN lecture). Then we discuss, that we will learn about three main concepts today: sequence-to-sequence models, attention mechanism and transformer. The first two are needed to understand the concept of the transformer. You can prepare the original papers and show them to the attendees.
Seq2seq: we just briefly discuss the main concepts. The difference between the teacher forcing (training) and instance-by-instance (inferance) should be emphasized.
The source codes should be discussed in details, line-by-line, so the concept can be understood by the students in a code level.
In the second half of the lecture the transformer architecture is introduced. The core elements are discussed seperately.
If you have some time left at the end of the lecture, you can open the TensorFlow tutorial on transformer (link on this page and in the slides too).
| Duration (Min) | Description |
|---|---|
| 5 | Sequential data introduction |
| 7.5 | Sequence-to-sequence models |
| 7.5 | Attention mechanism |
| 15 | Source codes |
| 20 | Transformer |
| 5 | Summary and conclusions |
Click here for an overview of all lesson plans of the master human centred AI
Please visit the home page of the consortium HCAIM
 |
The Human-Centered AI Masters programme was co-financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068. The materials of this learning event are available under CC BY-NC-SA 4.0
|
|
The HCAIM consortium consists of three excellence centres, three SMEs and four Universities
|
|
