Recently HF trainer was extended to support full fp16 eval via --fp16_full_eval. I'd have expected it to be either equal or faster than eval with fp32 model, but surprisingly I have noticed a 25% slowdown when using it.

This may or may not impact deepspeed as well, which also runs eval in fp16, but we can't compare it to a baseline, since it only runs fp16.

I wonder if someone would like t

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.

You can either:

1. Suggest a new feature by leaving a comment.
2. Vote for a feature request with 👍 or be against with 👎. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
3. Tell us that

🐛 Bug

Please reproduce using the BoringModel

Currently, the progress bar doesn't make distinction between running in a real terminal or being piped to logs.

When Lightning run in logs, it produces a prog_bar line for each change.

<img width="1659" alt="Screenshot 2021-03-23 at 09 10 40" src="https://user-images.git

Change tensor.data to tensor.detach() due to
pytorch/pytorch#6990 (comment)
tensor.detach() is more robust than tensor.data.

Add a new API for converting a model to external data. Today the conversion happens in 2 steps
external_data_helper.convert_model_to_external_data(<model>, <all_tensors_to_one_file>, <size_threshold>) save_model(model, output_path)
We want to add another api which combines the 2 steps
`
save_model_to_external_data(, <output_

While setting train_parameters to False very often we also may consider disabling dropout/batchnorm, in other words, to run the pretrained model in eval mode.
We've done a little modification to PretrainedTransformerEmbedder that allows providing whether the token embedder should be forced to eval mode during the training phase.

Do you this feature might be handy? Should I open a PR?

I'm using mxnet to do some work, but there is nothing when I search the mxnet trial and example.

Please can you train ghostnet.
(i don't have the imagenet dataset)

Is it possible to run this on a (recent) Mac, which does not support CUDA? I would have guessed setting --GPU 0 would not attempt to call CUDA, but it fails.

File "/Users/../Desktop/bopbtl/venv/lib/python3.7/site-packages/torch/cuda/__init__.py", line 61, in _check_driver
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enable