Create your own GitHub profile
Sign up for your own profile on GitHub, the best place to host code, manage projects, and build software alongside 50 million developers.
Sign up
Popular repositories
675 contributions in the last year
Contribution activity
July 1, 2020
dpressel has no activity
yet for this period.
June 2020
Created a pull request in dpressel/mead-baseline that received 1 comment
WIP: add tensorboard logging to pytorch pretrain-tlm
also, this adds a logger kwarg to the CompositeLRScheduler which allows us to pass in some object that supports the add_scalar signature provided b…
+20
−3
•
1
comment
- Feature/transformer disc
- changes the loss to match the discription in the paper
- option to mlm masking preserve non [MASK] x in y
- Feature/add causal preproc
- Feature/more utils in 8mi
- Feature/add pretrain clm tf
- WIP: updates the pretrain paired to work as expected
- Feature/cleanup gen-discrim training
- improve TB logging in both PyTorch and TF
- convert to NPZ option after a single training step
- Fix/match tlm def args
- TF example LRS was ignoring the LR on cosine decay
- set global_step in eager mode to iterations
- update base params
- support no tpu ep, fix cycle/par call mismatch
- adds shuffling to TF pretrain TLM
- add options to dropout the output a la bert
- were allowing 2 paths, update->step + update/step
- adds shuffling to distributed training
- dont distribute validation set in Pytorch TLMs
- Feature/save tf tlm npz
- switch to URL-capable function
- Feature/cleanup tf sample
20
contributions
in private repositories
Jun 2 – Jun 9