Administrative information

Title	Model Compression
Duration	150 min
Module	C
Lesson Type	Practical
Focus	Technical - Future AI
Topic	Advances in ML models through a HC lens - A result Oriented Study

Keywords

model compression, pruning, quantization, knowledge distillation,

Learning Goals

Understand how to implement techniques of model compression
Grasp the advantages of pruning, quantization and knowledge distillation
Becoming familiar with a high-level framework like TensorFlow

Expected Preparation

Learning Events to be Completed Before

Lecture: Model Compression - Edge Computing

Obligatory for Students

Basic understanding of model compression concepts and techniques
Basic understanding of how the performance of machine and deep learning models can be evaluated (e.g. accuracy, precision and recall, F score)
Knowledge of the Python programming language

Optional for Students

Knowledge of the TensorFlow framework

References and background for students

Knowledge of machine learning and neural networks theory

Recommended for Teachers

Recall knowledge of the TensorFlow framework and Python programming language
Provide a practical view on the implementations needed to leverage model compression techniques
Propose pop-up quizzes

Lesson materials

The materials of this learning event are available under CC BY-NC-SA 4.0.

Instructions for Teachers

Give a brief overview of Tensorflow 2.x
Use Google Colab as working Jupyter Notebook for practical application
Students must use the indicated time allocated for each task.
Task 1 to Task 4 should be completed before the remaining tasks are assigned.

Outline

Duration	Description	Concepts	Activity	Material
0-10 min	Introduction to tools used and how to make hands dirty in a second	Tools introduction	Introduction to main tools
10-80 min	[Task 1 - Task 3] Training a model and then? How to apply pruning and quantization to working models and compare performances	Pruning & Quantization	Practical session and working examples	Colab Notebook
80-140 min	[Task 4 - Task 6] When could be knowledge distillation useful? How to distill knowledge from teacher to student	Knowledge Distillation	Practical session and working examples	Colab Notebook
140-150 min	Conclusion, questions and answers	Summary	Conclusions

More information

Click here for an overview of all lesson plans of the master human centred AI

Please visit the home page of the consortium HCAIM

Acknowledgements

The Human-Centered AI Masters programme was co-financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.

The materials of this learning event are available under CC BY-NC-SA 4.0

The HCAIM consortium consists of three excellence centres, three SMEs and four Universities