Quantization vs Pruning vs Distillation: Optimizing NNs for Inference

Try Voice Writer - speak your thoughts and let AI handle the grammar: https://voicewriter.io

Four techniques to optimize the speed of your model's inference process:
0:38 - Quantization
5:59 - Pruning
9:48 - Knowledge Distillation
13:00 - Engineering Optimizations

References:

LLM Inference Optimization blog post: https://lilianweng.github.io/posts/2023-01-10-inference-optimization/

How to deploy your deep learning project on a budget: https://luckytoilet.wordpress.com/2023/06/20/how-to-deploy-your-deep-learning-side-project-on-a-budget/

Efficient deep learning survey paper: https://arxiv.org/abs/2106.08962

SparseDNN: https://arxiv.org/abs/2101.07948