Popular repositories

1. gptq gptq Public

   Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

   Python 1.7k 128

2. sparsegpt sparsegpt Public

   Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

   Python 601 76

3. marlin marlin Public

   FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

   Python 249 17

4. qmoe qmoe Public

   Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

   Python 242 21

5. QUIK QUIK Public

   Repository for the QUIK project, enabling the use of 4bit kernels for generative inference

   C++ 151 10

6. OBC OBC Public

   Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning".

   Python 82 11