distilbert

A smaller, faster, cheaper, and lighter version of BERT. It keeps 95% of BERT's performance while being 60% smaller and 60% faster.