In machine learning, knowledge distillation or model distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models ) have more knowledge capacity than small models, this capacity might not be fully utilized.
In machine learning, knowledge distillation or model distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models ) have more knowledge capacity than small models, this capacity might not be fully utilized.
Source: Wikipedia "Knowledge distillation" · CC BY-SA 4.0
Share this article: X · Bluesky