Skip to content

Issues: openai/tiktoken

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

22 Open 141 Closed
22 Open 141 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Plugins found: ['tiktoken_ext.openai_public']
#218 opened Nov 26, 2023 by voghoei
ERROR: Could not build wheels for tiktoken, which is required to install pyproject.toml-based projects
#215 opened Nov 18, 2023 by 2anirban
7
Can't install tiktoken in Python 3.12
#205 opened Oct 9, 2023 by pamelafox
5
tiktoken error on AWS lambda
#198 opened Sep 21, 2023 by vicky141998
2
tiktoken fails with Rust 1.60 since memchr 2.6.3 got released
#197 opened Sep 20, 2023 by jhgoebbert
Tiktoken's number of tokens does not match the number of tokens perceived by the gpt-3.5-turbo API
#196 opened Sep 20, 2023 by viswavi
Very slow for inputs like "a" * 100000
#195 opened Sep 18, 2023 by vvolhejn
8
Adding a __version__ attribute
#190 opened Sep 10, 2023 by rasbt
1
4
from tiktoken import _tiktoken is causing circular dependency on AWS lambda
#184 opened Sep 4, 2023 by akshat-g
4
Please commit your Cargo.lock file in the repository
#179 opened Aug 17, 2023 by charles-dyfis-net
1
tiktoken_ext.__pycache__ Issue On Amazone Lambda
#169 opened Jul 22, 2023 by FarzamHejaziK
1
ImportError: PyO3 modules may only be initialized once per interpreter process
#141 opened Jun 8, 2023 by jackyhzzj
8
Any introduction about api encode_with_unstable?
#137 opened May 26, 2023 by fseasy
1
Fix missing line numbers in stack traces
#103 opened Apr 13, 2023 by christopherhesse
FAQ (please read before opening a new issue)
#98 opened Apr 5, 2023 by hauntsaninja
Unofficial bindings / ports in other languages
#97 opened Apr 5, 2023 by hauntsaninja
10
Add im_start / im_end in cl100k_base
#48 opened Mar 5, 2023 by spolu
4
Thread Panic when decoding token id 100256 and others with cl100k_base tokenizer
#47 opened Mar 4, 2023 by minimaxir
1
Performance ideas
#33 opened Feb 20, 2023 by christopher-hesse
6
Tiktoken not published to cargo
#24 opened Jan 24, 2023 by zurawiki
1
11
NPM package
#22 opened Jan 22, 2023 by atgctg
7
Extremely Long Text results in PanicException, which is hard to catch in python code
#15 opened Dec 21, 2022 by codedecde
1
ProTip! Find all open issues with in progress development work with linked:pr.