Insights: lm-sys/FastChat

24 Pull requests merged by 12 people

• Improve sampling & Print worker id & Add Guanaco

  #1653 merged Jun 10, 2023

• Add Manticore

  #1649 merged Jun 10, 2023

• Adjust sampling weights & Fix palm-2

  #1648 merged Jun 9, 2023

• Add wizardlm

  #1647 merged Jun 9, 2023

• Fix GPTQ

  #1646 merged Jun 9, 2023

• Add fastest gptq 4bit inference support

  #1209 merged Jun 9, 2023

• Add Github Action for black formatting check

  #1539 merged Jun 9, 2023

• Update discord link

  #1645 merged Jun 9, 2023

• Release v0.2.12

  #1643 merged Jun 9, 2023

• [Fix] Fixed the mismatch issue between conv and model

  #1528 merged Jun 9, 2023

• Minor style fixes

  #1638 merged Jun 9, 2023

• Add model list reload support to gradio_web_server_multi

  #1487 merged Jun 9, 2023

• Add support for GPT4All-13B-Snoozy

  #1283 merged Jun 9, 2023

• Add warning when training with flash attention

  #1491 merged Jun 9, 2023

• Enable input grad for gradient checkpointing in `train_lora`

  #1489 merged Jun 9, 2023

• add api-key support

  #1520 merged Jun 9, 2023

• Add support for Gradio Webserver Authentication

  #1591 merged Jun 9, 2023

• Update OpenAI API and begin diverging.

  #1536 merged Jun 9, 2023

• fix chatglm-6b:when use /v1/completions ,IndexError: string index out of range

  #1625 merged Jun 9, 2023

• add ability for workers to advertise multiple model names

  #1517 merged Jun 9, 2023

• Count the tokens/second when using --debug

  #1573 merged Jun 9, 2023

• Fix missing controller address in docker-compose.yml

  #1633 merged Jun 9, 2023

• use python3.9 in dockerfile

  #1634 merged Jun 9, 2023

• fix zero3 save problem with minimum change

  #1457 merged Jun 5, 2023

4 Pull requests opened by 4 people

• Add Support For Robin-7b Model, a chatbot finetuned by HKUST

  #1621 opened Jun 6, 2023

• feat: add config file

  #1650 opened Jun 10, 2023

• Fix-dolly-8-bit

  #1656 opened Jun 11, 2023

• Add Korean model.

  #1658 opened Jun 11, 2023

20 Issues closed by 10 people

• pad token comparison

  #1662 closed Jun 11, 2023

• fine-tuning error: ModuleNotFoundError: No module named 'flash_attn'

  #1664 closed Jun 11, 2023

• vicuna model file is twice the size of LLAMA. Is it Correct?

  #1663 closed Jun 11, 2023

• TypeError: __init__() got an unexpected keyword argument 'torch_dtype'

  #1651 closed Jun 11, 2023

• An error is reported during fine-tuning, how to solve it

  #1654 closed Jun 11, 2023

• [Chatbot Arena] Add Wizard-Vicuna-Uncensored 13B and 30B models

  #1505 closed Jun 10, 2023

• add guanaco series models

  #1563 closed Jun 10, 2023

• Error while saving model after training flan-t5-xxl on train_flant5.py

  #1642 closed Jun 10, 2023

• Support for GPTQ-LLAMA

  #452 closed Jun 9, 2023

• Please add GPT4All-L Snoozy 13B to bot arena

  #863 closed Jun 9, 2023

• Add repetition_penalty to OpenAI API compatible generation endpoints.

  #1537 closed Jun 9, 2023

• There is a problem when using chatglm-6b to run the worker and use langchain to access the api

  #1385 closed Jun 9, 2023

• Tokens/second reading when --debug

  #1268 closed Jun 9, 2023

• demo is not working locally

  #1632 closed Jun 8, 2023

• Starting CLI returns index out of range error

  #1600 closed Jun 8, 2023

• train_lora.py Token indices sequence length is longer than the specified maximum sequence length for this model

  #1592 closed Jun 7, 2023

• api how to use history

  #1622 closed Jun 7, 2023

• Vicuna-7b Multi-gpu finetuning, stuck on the first evaluation

  #1614 closed Jun 6, 2023

• Where is the file ? state_of_the_union.txt

  #1616 closed Jun 6, 2023

• API server error with langchain LangChain with FastChat(openai.error.APIError: Invalid response object from API: '{"object":"error")

  #1546 closed Jun 5, 2023

24 Issues opened by 23 people

• support Vicuna finetune with qLoRA

  #1657 opened Jun 11, 2023

• How to enable batch evaluation? I got RuntimeError: CUDA error: device-side assert triggered

  #1655 opened Jun 10, 2023

• FastChat with API using only one processor core on CPU for output generation

  #1652 opened Jun 10, 2023

• ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 1 (pid: 5448) of binary: /home/win10-ubuntu/anaconda3/envs/vicuna-7b/bin/python3.10

  #1644 opened Jun 9, 2023

• report error while i execute `python -m fastchat.serve.openai_api_server --host localhost --port 8000`

  #1641 opened Jun 9, 2023

• How to use a single GPU for training?

  #1640 opened Jun 9, 2023

• Model worker keeps on registering and gets de registered

  #1639 opened Jun 9, 2023

• Hello everyone, can we perform Lora or other fine-tuning based on the Vicuna model? How much graphics memory is required?

  #1637 opened Jun 9, 2023

• what ML library does this project use?

  #1636 opened Jun 9, 2023

• Split conversations with IJSON.

  #1635 opened Jun 8, 2023

• Support Semantic Kernel with FastChat

  #1631 opened Jun 8, 2023

• ValueError: Tokenizer class GPTNeoXTokenizer does not exist or is not currently imported.

  #1630 opened Jun 8, 2023

• where can find cacheflow

  #1629 opened Jun 8, 2023

• It's very fuck!!! like this:

  #1628 opened Jun 8, 2023

• AttributeError: module 'torch.cuda' has no attribute 'OutOfMemoryError'

  #1627 opened Jun 8, 2023

• fastchat.serve.model_worker --device cpu only uses one CPU Thread for token generation.

  #1626 opened Jun 7, 2023

• module 'fastchat' has no attribute 'load_model'

  #1623 opened Jun 7, 2023

• Conversation templates Api

  #1618 opened Jun 6, 2023

• Support logprob in OpenAI API

  #1615 opened Jun 6, 2023

• Does vicuna-13b/vicuna-7b provides flash-attention implementation during inference using this repo. If yes, where is the implementation?

  #1610 opened Jun 5, 2023

• fschat=0.2.11 in pypi is offical version or not?

  #1608 opened Jun 5, 2023

• Language distribution of ShareGPT 70K conversation dataset for FastChat T5

  #1607 opened Jun 5, 2023

• RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

  #1606 opened Jun 5, 2023

• Fine-tuning Vicuna-7B with LoRa?

  #1602 opened Jun 5, 2023

28 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Fixed bug in openai_api_server.py

  #1547 commented on Jun 9, 2023 • 6 new comments

• leave only 45 conversations in dummy.json result in error

  #1097 commented on Jun 9, 2023 • 3 new comments

• [Chatbot Arena] Add Falcon 40B model

  #1504 commented on Jun 10, 2023 • 3 new comments

• Fine-tuning Falcon

  #1588 commented on Jun 10, 2023 • 3 new comments

• Add LoraAdapter to model_adapter.py

  #1482 commented on Jun 9, 2023 • 3 new comments

• Any plans for 30B version?

  #170 commented on Jun 5, 2023 • 2 new comments

• 2 node speed is not faster than 1 node

  #1153 commented on Jun 7, 2023 • 2 new comments

• How to fine tune vicuna-7b with A40

  #1296 commented on Jun 8, 2023 • 2 new comments

• Training new Vicuna based on fully open-source OpenLLaMA

  #1154 commented on Jun 5, 2023 • 1 new comment

• Text garbage

  #1321 commented on Jun 5, 2023 • 1 new comment

• Trying to load a safetensors file

  #1530 commented on Jun 6, 2023 • 1 new comment

• ValueError: Asking to pad but the tokenizer does not have a padding token.

  #1534 commented on Jun 7, 2023 • 1 new comment

• NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.

  #1527 commented on Jun 9, 2023 • 1 new comment

• python3 -m fastchat.serve.model_worker returns status_code 403

  #1282 commented on Jun 9, 2023 • 1 new comment

• api_server runs too slowly

  #1499 commented on Jun 9, 2023 • 1 new comment

• I have some questions, such as how to create a public link that can open the dialog interface in your browser

  #1477 commented on Jun 9, 2023 • 1 new comment

• How to use lora to train the 30b model on multiple machines and multiple cards?

  #615 commented on Jun 9, 2023 • 1 new comment

• torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

  #1200 commented on Jun 9, 2023 • 1 new comment

• Improve SSE User Experience

  #1223 commented on Jun 9, 2023 • 1 new comment

• can i use this code to conduct self-supervised pre-training based on llama-7b

  #1556 commented on Jun 5, 2023 • 0 new comments

• Add scaled_dot_product_attention to replace flash attention

  #177 commented on Jun 9, 2023 • 0 new comments

• Added Notebooks for FastChat

  #427 commented on Jun 9, 2023 • 0 new comments

• Issue#270 add CI to support release and publish

  #832 commented on Jun 9, 2023 • 0 new comments

• Byte deltas

  #1045 commented on Jun 9, 2023 • 0 new comments

• Add xformer and support training on V100s

  #1255 commented on Jun 9, 2023 • 0 new comments

• Add callback on save for LoRA

  #1280 commented on Jun 9, 2023 • 0 new comments

• Update input args (require model_path if model_name provided)

  #1432 commented on Jun 9, 2023 • 0 new comments

• CAMEL 13B Model Support

  #1593 commented on Jun 9, 2023 • 0 new comments