Administrative information
Title
|
Model Compression |
![](/userfiles/ded16fad2a17a5781460974accb8e136815acf75.png) |
Duration |
150 min |
Module |
C |
Lesson Type |
Practical |
Focus |
Technical - Future AI |
Topic |
Advances in ML models through a HC lens - A result Oriented Study |
|
Keywords
model compression, pruning, quantization, knowledge distillation,
Learning Goals
- Understand how to implement techniques of model compression
- Grasp the advantages of pruning, quantization and knowledge distillation
- Becoming familiar with a high-level framework like TensorFlow
Expected Preparation
Learning Events to be Completed Before
Obligatory for Students
- Basic understanding of model compression concepts and techniques
- Basic understanding of how the performance of machine and deep learning models can be evaluated (e.g. accuracy, precision and recall, F score)
- Knowledge of the Python programming language
Optional for Students
- Knowledge of the TensorFlow framework
References and background for students
- Knowledge of machine learning and neural networks theory
Recommended for Teachers
- Recall knowledge of the TensorFlow framework and Python programming language
- Provide a practical view on the implementations needed to leverage model compression techniques
- Propose pop-up quizzes
Lesson materials
The materials of this learning event are available under CC BY-NC-SA 4.0.
Instructions for Teachers
- Give a brief overview of Tensorflow 2.x
- Use Google Colab as working Jupyter Notebook for practical application
- Students must use the indicated time allocated for each task.
- Task 1 to Task 4 should be completed before the remaining tasks are assigned.
Outline
Duration |
Description |
Concepts |
Activity |
Material |
0-10 min |
Introduction to tools used and how to make hands dirty in a second |
Tools introduction |
Introduction to main tools |
|
10-80 min |
[Task 1 - Task 3] Training a model and then? How to apply pruning and quantization to working models and compare performances |
Pruning & Quantization |
Practical session and working examples |
Colab Notebook |
80-140 min |
[Task 4 - Task 6] When could be knowledge distillation useful? How to distill knowledge from teacher to student |
Knowledge Distillation |
Practical session and working examples |
Colab Notebook |
140-150 min |
Conclusion, questions and answers |
Summary |
Conclusions |
|
More information
Click here for an overview of all lesson plans of the master human centred AI
Please visit the home page of the consortium HCAIM
Acknowledgements
![](/userfiles/85b144cadea12e28dfbd499b4e42fb4ec17e0ef7.png) |
The Human-Centered AI Masters programme was co-financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.
The materials of this learning event are available under CC BY-NC-SA 4.0
|
The HCAIM consortium consists of three excellence centres, three SMEs and four Universities
![HCAIM Consortium](/userfiles/3e488da2074635154b03fa625b9c1387996b14d0.png)
|