Build software better, together

microsoft / nni

Open

ChannelDependency does not handle concat properly

1

pkubik commented Mar 14, 2022

Describe the issue:
During computing Channel Dependencies reshape_break_channel_dependency does following code to ensure that the number of input channels equals the number of output channels:

in_shape = op_node.auxiliary['in_shape']
out_shape = op_node.auxiliary['out_shape']
in_channel = in_shape[1]
out_channel = out_shape[1]
return in_channel != out_channel

This is correct

Does nni contain some new pruning base methods？

6

Open

nnictl experiment delete -all fail for stopped expriment

1

Find more good first issues

dkozlov / awesome-knowledge-distillation

Star

Awesome Knowledge Distillation

deep-learning knowledge-distillation teacher-student knowledge-transfer co-training model-compression distillation kd knowldge-distillation distillation-model model-distillation

Updated Mar 12, 2022

Tencent / PocketFlow

Star

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.

computer-vision deep-learning mobile-app automl model-compression

Updated Jan 15, 2021
Python

huawei-noah / Pretrained-Language-Model

Star

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

pretrained-models quantization knowledge-distillation model-compression large-scale-distributed

Updated Mar 31, 2022
Python

huawei-noah / CV-Backbones

Star

CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.

tensorflow pytorch transformer imagenet convolutional-neural-networks pretrained-models cvpr model-compression efficient-inference ghostnet vision-transformer

Updated Mar 28, 2022
Python

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

pytorch pruning convolutional-networks quantization xnor-net tensorrt model-compression bnn neuromorphic-computing group-convolution onnx network-in-network tensorrt-int8-python dorefa twn network-slimming integer-arithmetic-only quantization-aware-training post-training-quantization batch-normalization-fuse

Updated Oct 6, 2021
Python

FLHonker / Awesome-Knowledge-Distillation

Star

Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。

deep-learning transfer-learning model-compression distillation kd knowldge-distillation

Updated Apr 11, 2022

he-y / Awesome-Pruning

Star

A curated list of neural network pruning resources.

awesome-list pruning model-compression model-acceleration

Updated Oct 31, 2021

microsoft / NeuronBlocks

Star

NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego

natural-language-processing deep-learning text-classification dnn pytorch artificial-intelligence question-answering knowledge-distillation sequence-labeling text-matching qna model-compression

Updated Mar 16, 2021
Python

peterliht / knowledge-distillation-pytorch

Star

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

deep-neural-networks computer-vision pytorch knowledge-distillation cifar10 dark-knowledge model-compression

Updated Mar 11, 2022
Python

tensorflow / model-optimization

Star

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

machine-learning sparsity compression deep-learning tensorflow optimization keras ml pruning quantization model-compression quantized-training quantized-neural-networks quantized-networks

Updated Apr 14, 2022
Python

yihui-he / channel-pruning

Star

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

deep-neural-networks acceleration image-classification image-recognition object-detection model-compression channel-pruning

Updated Feb 28, 2022
Python

AberHu / Knowledge-Distillation-Zoo

Star

Pytorch implementation of various Knowledge Distillation (KD) methods.

knowledge-distillation teacher-student knowledge-transfer model-compression distillation kd kd-methods

Updated Nov 25, 2021
Python

PaddlePaddle / PaddleSlim

Star

PaddleSlim is an open-source library for deep model compression and architecture search.

pruning quantization nas knowledge-distillation evolution-strategy model-compression neural-architecture-search hyperparameter-search autodl

Updated May 7, 2022
Python

guan-yuan / awesome-AutoML-and-Lightweight-Models

Star

A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.

tensorflow pytorch hyperparameter-optimization awesome-list quantization nas automl model-compression neural-architecture-search meta-learning architecture-search quantized-training model-acceleration automated-feature-engineering quantized-neural-network

Updated Jun 19, 2021

MingSun-Tse / EfficientDNNs

Star

Collection of recent methods on (deep) neural network compression and acceleration.

deep-neural-networks deep-learning knowledge-distillation model-compression network-pruning

Updated Mar 31, 2022

cnkuangshi / LightCTR

Star

Lightweight and Scalable framework that combines mainstream algorithms of Click-Through-Rate prediction based computational DAG, philosophy of Parameter Server and Ring-AllReduce collective communication.

distributed-systems machine-learning deep-learning factorization-machines computational-graphs parameter-server model-compression

Updated Jun 17, 2019
C++

VainF / Torch-Pruning

Star

Pruning channels for model acceleration

pytorch pruning model-compression network-pruning

Updated May 4, 2022
Python

lhyfst / knowledge-distillation-papers

Star

knowledge distillation papers

paper reading-list knowledge-distillation dark-knowledge model-compression

Updated May 7, 2022

iamhankai / ghostnet.pytorch

Star

[CVPR2020] GhostNet: More Features from Cheap Operations

pytorch convolutional-neural-networks model-compression fbnet mobilenetv3

Updated Aug 8, 2020
Python

huawei-noah / Efficient-Computing

Star

Efficient-Computing

knowledge-distillation model-compression

Updated Dec 8, 2021
Python

he-y / filter-pruning-geometric-median

Star

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (CVPR 2019 Oral)

pytorch pruning model-compression

Updated Jun 17, 2021
Python

SforAiDl / KD_Lib

Star

Open

Benchmarking Pruning and Quantization

avishreekh commented May 7, 2021

We also need to benchmark the Lottery-tickets Pruning algorithm and the Quantization algorithms. The models used for this would be the student networks discussed in #105 (ResNet18, MobileNet v2, Quantization v2).

Pruning (benchmark upto 40, 50 and 60 % pruned weights)