transformer
Here are 1,616 public repositories matching this topic...
-
Updated
Jul 25, 2021 - Jupyter Notebook
Bidirectional RNN
Is there a way to train a bidirectional RNN (like LSTM or GRU) on trax nowadays?
-
Updated
Dec 28, 2021 - Jupyter Notebook
-
Updated
Sep 10, 2021 - Python
chooses 15% of token
From paper, it mentioned
Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.
It means that 15% of token will be choose for sure.
From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.
PositionalEmbedding
-
Updated
Dec 16, 2021 - Python
-
Updated
Aug 25, 2021 - Python
-
Updated
Jan 6, 2022 - Python
-
Updated
Jan 6, 2022 - Python
-
Updated
Aug 4, 2021 - Jupyter Notebook
-
Updated
May 3, 2017 - Java
We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.
You can either:
- Suggest a new feature by leaving a comment.
- Vote for a feature request with
👍 or be against with👎 . (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!) - Tell us that you wo
-
Updated
Dec 12, 2021 - JavaScript
文档增加tokenizer类别及样例建议
欢迎您反馈PaddleNLP使用问题,非常感谢您对PaddleNLP的贡献!
在留下您的问题时,辛苦您同步提供如下信息:
- 版本、环境信息
1)PaddleNLP和PaddlePaddle版本:请提供您的PaddleNLP和PaddlePaddle版本号,例如PaddleNLP 2.0.4,PaddlePaddle2.1.1
2)系统环境:请您描述系统类型,例如Linux/Windows/MacOS/,python版本 - 复现信息:如为报错,请给出复现环境、复现步骤
paddle版本2.0.8 paddlenlp版本2.1.0
建议,能否在paddlenlp文档中,整理列出各个模型的tokenizer是基于什么类别的based,如bert tokenizer是word piece的,xlnet tokenizer是sentence piece的,以及对应的输入输出样例
关于一些具体建议
-
Updated
Jan 6, 2022 - Python
-
Updated
Jul 20, 2021 - Python
-
Updated
Jan 2, 2022
-
Updated
Jan 4, 2022 - Cuda
-
Updated
Jan 6, 2022 - Python
-
Updated
Jan 6, 2022 - C++
-
Updated
Nov 30, 2021 - Python
-
Updated
Dec 29, 2021 - Jupyter Notebook
-
Updated
Oct 14, 2021 - Python
-
Updated
Jun 14, 2021 - Jupyter Notebook
-
Updated
Aug 9, 2021 - Python
-
Updated
Dec 29, 2021 - Python
-
Updated
Dec 14, 2021 - Jupyter Notebook
-
Updated
Dec 16, 2021 - Java
-
Updated
Nov 26, 2021 - Python
Improve this page
Add a description, image, and links to the transformer topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the transformer topic, visit your repo's landing page and select "manage topics."
Fast Tokenizer for DeBERTA-V3 and mDeBERTa-V3
Motivation
DeBERTa V3 is an improved version of DeBERTa. With the V3 version, the authors also released a multilingual model "mDeBERTa-base" that outperforms XLM-R-base. However, DeBERTa V3 currently lacks a FastTokenizer implementation which makes it impossible to use with some of the example scripts (They require a Fa