🚀 Feature request

Currently, the EncoderDecoderModel class in PyTorch automatically creates the decoder_input_ids based on the labels provided by the user (similar to how this is done for T5/BART). This should also be implemented for TFEncoderDecoderModel, because currently users should manually provide decoder_input_ids to the model.

One can take a look at the TF implementation

Proposed refactoring or deprecation

Motivation

Lightning has a utility defined for all gather with gradients here: https://github.com/PyTorchLightning/pytorch-lightning/blob/d515bcac969c2a485ada673e302bfac51f142331/pytorch_lightning/utilities/distributed.py#L200-L222

However, this is already available in torch dist

Change tensor.data to tensor.detach() due to
pytorch/pytorch#6990 (comment)
tensor.detach() is more robust than tensor.data.

🙌🙌🙌🙌🙌

New Operator

Describe the operator

Why is this operator necessary? What does it accomplish?

This is a frequently used operator in tensorflow/keras

Can this operator be constructed using existing onnx operators?

If so, why not add it as a function?

I don't know.

Is this operator used by any model currently? Which one?

Are you willing to contribute it?

Motivated by huggingface/transformers#12789 in Transformers, one welcoming change would be replacing assertions with proper exceptions. The only type of assertions we should keep are those used as sanity checks.

Currently, there is a total of 87 files with the assert statements (located under datasets and src/datasets), so when working on this, to manage the PR s

Is your feature request related to a problem? Please describe.
I typically used compressed datasets (e.g. gzipped) to save disk space. This works fine with AllenNLP during training because I can write my dataset reader to load the compressed data. However, the predict command opens the file and reads lines for the Predictor. This fails when it tries to load data from my compressed files.