Awesome Knowledge Distillation
-
Updated
May 30, 2021
#
knowledge-distillation
Here are 174 public repositories matching this topic...
A treasure chest for visual recognition powered by PaddlePaddle
image-classification
image-recognition
pretrained-models
knowledge-distillation
product-recognition
autoaugment
cutmix
randaugment
gridmask
deit
repvgg
swin-transformer
image-retrieval-system
-
Updated
Oct 13, 2021 - Python
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
-
Updated
Oct 12, 2021 - Python
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
natural-language-processing
deep-learning
text-classification
dnn
pytorch
artificial-intelligence
question-answering
knowledge-distillation
sequence-labeling
text-matching
qna
model-compression
-
Updated
Mar 16, 2021 - Python
Improving Convolutional Networks via Attention Transfer (ICLR 2017)
-
Updated
Jul 11, 2018 - Jupyter Notebook
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
deep-neural-networks
computer-vision
pytorch
knowledge-distillation
cifar10
dark-knowledge
model-compression
-
Updated
Sep 7, 2021 - Python
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning
quantization
nas
knowledge-distillation
evolution-strategy
model-compression
neural-architecture-search
hyperparameter-search
autodl
-
Updated
Oct 13, 2021 - Python
Pytorch implementation of various Knowledge Distillation (KD) methods.
knowledge-distillation
teacher-student
knowledge-transfer
model-compression
distillation
kd
kd-methods
-
Updated
Oct 13, 2021 - Python
EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
-
Updated
Sep 15, 2021 - Python
利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码
flask
deployment
random-forest
svm
cnn
pytorch
image-classification
densenet
resnet
knn
knowledge-distillation
resnext
label-smoothing
-
Updated
Apr 13, 2021 - Jupyter Notebook
knowledge distillation papers
-
Updated
Jun 9, 2020
A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆 20 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
nlp
natural-language-processing
glue
pytorch
transformer
imagenet
image-classification
coco
object-detection
semantic-segmentation
knowledge-distillation
cifar10
pascal-voc
cifar100
google-colab
colab-notebook
-
Updated
Sep 27, 2021 - Python
Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
-
Updated
Jun 2, 2021 - Python
Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
deep-learning
human-pose-estimation
mpii-dataset
knowledge-distillation
mscoco-keypoint
coco-keypoints-detection
fast-pose-distillation
-
Updated
Oct 12, 2021 - Cuda
Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
-
Updated
Jun 23, 2020 - Python
Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
-
Updated
Sep 8, 2021 - Python
(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)
python
distributed-systems
machine-learning
data-mining
machine-learning-algorithms
machine-learning-library
outlier-detection
knowledge-distillation
anomaly-detection
-
Updated
Oct 2, 2021 - Python
Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
-
Updated
May 17, 2021 - Python
Infrastructures™ for Machine Learning Training/Inference in Production.
kubernetes
machine-learning
apache-spark
deep-learning
artificial-intelligence
awesome-list
pruning
quantization
knowledge-distillation
deep-learning-framework
model-compression
apache-arrow
federated-learning
machine-learning-systems
apache-mesos
-
Updated
May 24, 2019
Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)
-
Updated
Nov 21, 2019 - Python
Using Teacher Assistants to Improve Knowledge Distillation: https://arxiv.org/pdf/1902.03393.pdf
-
Updated
Oct 3, 2019 - Python
Intel® Neural Compressor (formerly known as Intel® Low Precision Optimization Tool), targeting to provide unified APIs for network compression technologies, such as low precision, sparsity, pruning, knowledge distillation, cross different deep learning frameworks to purse best inference performance.
sparsity
pruning
quantization
knowledge-distillation
auto-tuning
low-precision
quantization-aware-training
post-training-quantization
-
Updated
Oct 11, 2021 - Python
A large scale study of Knowledge Distillation.
-
Updated
Apr 19, 2020 - Python
[CVPR'20] PyTorch code for our paper "Collaborative Distillation for Ultra-Resolution Universal Style Transfer"
-
Updated
Sep 8, 2021 - Python
Training & evaluation library for text-based neural re-ranking and dense retrieval models built with PyTorch
-
Updated
Jul 26, 2021 - Python
Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers
nlp
naacl
notes
presentation
slides
acl
presentations
conversation
gan
generation
summarization
emnlp
knowledge-distillation
data-augmentation
aaai
meta-learning
graph-neural-networks
gnn
pretrain
non-autoregressive
-
Updated
Oct 3, 2021
Knowledge Distillation using Tensorflow
-
Updated
Aug 12, 2019 - Python
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images
machine-learning
deep-neural-networks
deep-learning
tensorflow
knowledge-distillation
facial-expression-recognition
model-compression
-
Updated
Mar 7, 2021 - Python
Improve this page
Add a description, image, and links to the knowledge-distillation topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the knowledge-distillation topic, visit your repo's landing page and select "manage topics."
We also need to benchmark the Lottery-tickets Pruning algorithm and the Quantization algorithms. The models used for this would be the student networks discussed in #105 (ResNet18, MobileNet v2, Quantization v2).
Pruning (benchmark upto 40, 50 and 60 % pruned weights)
Quantization