TokenScript schema, specs and paper
-
Updated
Mar 31, 2022 - HTML
#
tokenisation
Here are 5 public repositories matching this topic...
Open morphology for Finnish
-
Updated
Oct 2, 2021 - Python
A Python 3 module that provides functions for splitting identifiers found in source code files.
python
machine-learning
source-code
splitter
camelcase
identifier
mining-software-repositories
split-string
tokenisation
splitting-identifiers
-
Updated
Dec 4, 2020 - Python
Built a complete search engine by creating an Inverted Index on the Wikipedia corpus ( of 2018 with size 72 GB). That gives you top search result related to given query words.
search
wikipedia
mergesort
inverted-index
response-time
stemming
tokenisation
search-engine-algorithm
stop-words
-
Updated
Oct 1, 2020 - Jupyter Notebook
A search engine is constructed to return customised recipes according to three sorting algorithms. Speed is improved by performing pre-processing and inverted index.
-
Updated
Nov 24, 2021 - Jupyter Notebook
Improve this page
Add a description, image, and links to the tokenisation topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the tokenisation topic, visit your repo's landing page and select "manage topics."