explosion / spacy-transformers

Star

Open

Inconsistent tokenizer between spacy and spacy transformer

1

lamthuy commented Apr 1, 2020

Hi,
When we try to tokenize the following sentence:

If we use spacy

a = spacy.load('en_core_web_lg')

doc = a("I like the link http://www.idph.iowa.gov/ohds/oral-health-center/coordinator")

list(doc)

We got

[I, like, the, link, http://www.idph.iowa.gov, /, ohds, /, oral, -, health, -, center, /, coordinator]

But if we use the Spacy transformer tokenizer:

How to fine tune weights of transformer layers

Open

Adding tagger, parser, ner in spacy_transformers pipeline

9

Find more good first issues →

ThilinaRajapakse / pytorch-transformers-classification

Star

Based on the Pytorch-Transformers library by HuggingFace. To be used as a starting point for employing Transformer models in text classification tasks. Contains code to easily train BERT, XLNet, RoBERTa, and XLM models for text classification.

natural-language-processing text-classification huggingface pytorch-transformers transformer-models

Updated May 9, 2020
Jupyter Notebook

kamalkraj / BERT-SQuAD

Star

SQuAD Question Answering Using BERT, PyTorch

pytorch question-answering pretrained-models bert bert-questionandanswering bert-qna-pretrained-models huggingface bert-squad

Updated Aug 24, 2019
Python

eagle705 / pytorch-bert-crf-ner

Star

KoBERT와 CRF로 만든 한국어 개체명인식기 (BERT+CRF based Named Entity Recognition model for Korean)

nlp natural-language-processing crf pytorch named-entity-recognition korean ner bert korean-nlp attention-visualization pytorch-implementation bert-bilstm-crf huggingface bert-crf kobert kobert-crf bert-bigru-crf

Updated Mar 2, 2020
Jupyter Notebook

labteral / ernie

Star

Simple State-of-the-Art BERT-Based Sentence Classification with Keras / TensorFlow 2. Built with HuggingFace's Transformers.

nlp natural-language-processing tensorflow keras transformers sentence-classification albert bert roberta bert-model transformer-architecture tensorflow2 bert-chinese bert-embeddings huggingface bert-models bert-as-service transformer-tensorflow2 distilbert huggingface-transformer

Updated Jun 20, 2020
Python

surbhardwaj / BERT-QnA-Squad_2.0_Finetuned_Model

Star

BERT which stands for Bidirectional Encoder Representations from Transformations is the SOTA in Transfer Learning in NLP.

download pytorch question-answering pretrained-models squad bert bert-model bert-questionandanswering bert-qna-pretrained-models huggingface bert-models bert-pytorch

Updated Mar 27, 2020
Python

nguyenvulebinh / vietnamese-electra

Star

Electra pre-trained model using Vietnamese corpus

natural-language-processing deep-learning vietnamese transformer language-model bert vietnamese-nlp fine-tuning electra tensorflow2 bert-embeddings huggingface

Updated Apr 24, 2020
Jupyter Notebook

manideep2510 / siamese-BERT-fake-news-detection-LIAR

Star

Triple Branch BERT Siamese Network for fake news classification on LIAR-PLUS dataset in PyTorch

machine-learning pytorch artificial-intelligence artificial-neural-networks fake-news liar fake-news-challenge bert-model language-understanding huggingface bert-models liar-plus bert-siamese

Updated Feb 9, 2020
Python

denocris / NLP-Workshop-MLMilan

Star

Advanced NLP Workshop: word-sense disambiguation with RoBERTa and text summarization with BART (Machine Learning Milan)

nlp machine-learning natural-language-processing deep-learning transformer machienlearning huggingface

Updated Jun 18, 2020
Jupyter Notebook

jetnew / DrFAQ

Star

Question Answering NLP Chatbot for Text Document Corpora, built using spaCy, huggingface’s BERT language model, ElasticSearch, Telegram Bot API, and hosted on Heroku.

elasticsearch natural-language-processing telegram-bot chatbot spacy question-answering bert huggingface