bart (bidirectional and auto-regressive transformers)

A denoising autoencoder for pretraining sequence-to-sequence models. BART is structurally similar to a traditional transformer encoder-decoder model, but is pre-trained like a denoising autoencoder.