tokenization
Here are 184 public repositories matching this topic...
-
Updated
Feb 13, 2020 - C++
OSX build notes have the following line
brew install automake berkeley-db4 libtool boost --c++11 miniupnpc openssl pkg-config protobuf python3 qt libevent
However, the boost --c++11 isn't a valid command anymore. Need to update it
-
Updated
May 2, 2020 - PHP
-
Updated
Oct 22, 2019 - Python
-
Updated
May 9, 2017 - C
-
Updated
May 5, 2020 - Python
The Transaction.md file doesn't contain enough details about its actual behavior.
-
Updated
May 19, 2020 - C++
-
Updated
May 18, 2020 - Python
It would be useful to add a sentence splitter, for instance, possibilities could be,
- Puntk sentence tokenizer from NLTK (needs pre-trained model)
- Unicode sentence boundaries from unicode-rs/unicode-segmentation#24 (doesn't need a pre-trained model)
- investigate spacy implementation (likely needs pre-traine
-
Updated
Mar 31, 2020 - Jupyter Notebook
-
Updated
Dec 20, 2019 - Rust
-
Updated
Aug 26, 2019
-
Updated
May 9, 2020 - Go
-
Updated
Feb 20, 2020 - Julia
-
Updated
Apr 18, 2020 - Python
-
Updated
Sep 13, 2017 - JavaScript
-
Updated
May 30, 2019 - Ruby
morphology_han-readings.py passes "北京大学生物系主任办公室内部会议" and prints out
{'hanReadings': [['Bei3-jing1-Da4-xue2'], null, ['zhu3-ren4'], ['ban4-gong1-shi4'], ['nei4-bu4'], ['hui4-yi4']]}
The element of the list, null, should be ['Sheng1-wu4'], i.e., "Biology."
-
Updated
Nov 23, 2019 - JavaScript
-
Updated
May 14, 2019 - Python
-
Updated
Sep 14, 2018 - TeX
-
Updated
May 16, 2020 - C++
-
Updated
May 20, 2020
-
Updated
May 19, 2020 - Go
-
Updated
Jul 17, 2019 - Java
-
Updated
Jan 15, 2020 - Jupyter Notebook
-
Updated
Aug 22, 2017 - Python
Improve this page
Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."
i'm a newbie in programming. I try to use this library. it's very useful for me.
I want to show centroid in K-means clustering. how to show it? thank u so much..