-
Updated
Oct 3, 2020 - Jupyter Notebook
transformer
Here are 816 public repositories matching this topic...
chooses 15% of token
From paper, it mentioned
Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.
It means that 15% of token will be choose for sure.
From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.
PositionalEmbedding
-
Updated
Jul 4, 2020 - Python
-
Updated
Sep 25, 2020 - Python
-
Updated
Aug 17, 2020 - Python
-
Updated
May 3, 2017 - Java
-
Updated
Oct 15, 2020 - JavaScript
-
Updated
Jun 1, 2020 - Jupyter Notebook
-
Updated
Feb 7, 2019 - Python
-
Updated
Sep 20, 2020 - Python
-
Updated
Oct 16, 2020 - Python
-
Updated
May 16, 2020 - Python
-
Updated
Sep 24, 2020 - Java
-
Updated
Jul 26, 2019 - Python
-
Updated
May 12, 2020 - Java
-
Updated
Oct 10, 2020 - Python
-
Updated
Sep 3, 2020 - Python
Hi, I am so interesting in your project, and wonder if you need contributor and how could I make my own contribution?
-
Updated
Jun 1, 2020
-
Updated
Nov 15, 2019 - Python
-
Updated
Jun 27, 2020 - Jupyter Notebook
-
Updated
May 30, 2017 - Java
-
Updated
Jul 16, 2020 - TypeScript
-
Updated
Oct 15, 2020 - Python
-
Updated
May 7, 2020 - Python
-
Updated
Jul 3, 2020 - Python
-
Updated
Oct 14, 2020
-
Updated
Oct 8, 2020 - JavaScript
Improve this page
Add a description, image, and links to the transformer topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the transformer topic, visit your repo's landing page and select "manage topics."
Is there a way to train a bidirectional RNN (like LSTM or GRU) on trax nowadays?