albert (a lite bert)

ALBERT is a version of BERT that reduces the number of model parameters without significantly decreasing performance. It uses factorized embedding parameterization and cross-layer parameter sharing.