Current implementation of Go binding can not specify options.

GPUOptions struct is in internal package. And go generate doesn't work for protobuf directory. So we can't specify GPUOptions for NewSession.

🚀 Add missing tokenizer test files

Several tokenizers currently have no associated tests. I think that adding the test file for one of these tokenizers could be a very good way to make a first contribution to transformers.

Tokenizers concerned

not yet claimed

none

claimed

• LED @nnlnr
• Flaubert @anmolsjoshi
• Longformer @tgadeliya
• Electra @rajathb

Followup to pytorch/pytorch#74955 (comment).

It turns out that that cmake version was just bad and we can now unpin cmake once again.

cc @seemethere @malfet @pytorch/pytorch-dev-infra

The implementation of the progress reporter, particularly the table generation, suffers from cluttered legacy code. The functions are long, messy, and pass around long argument lists. It is hard to unit test. We should clean this module up do make it easier to extend and modify.