Hi there!
Recently I've done a deep dive into MarianMT models, specifically to get OpusMT and Firefox translation models to work on Android under ONNX. Getting OpusMT/Firefoxs teacher models to work is relatively straight straightforward as HuggingFace's Transformers package supports the export of Marian models.
However, Firefox student models (which are a lot more efficient) are in TransformerRNN format. Would it be possible to train student models in Transformer structure? Would they still be relatively efficient?
For more context; In my ONNX implementation on Android I have tackled the mentioned shortcomings (missing beam search) by implementing a custom beam search algorithm.
Hi there!
Recently I've done a deep dive into MarianMT models, specifically to get OpusMT and Firefox translation models to work on Android under ONNX. Getting OpusMT/Firefoxs teacher models to work is relatively straight straightforward as HuggingFace's Transformers package supports the export of Marian models.
However, Firefox student models (which are a lot more efficient) are in TransformerRNN format. Would it be possible to train student models in Transformer structure? Would they still be relatively efficient?
For more context; In my ONNX implementation on Android I have tackled the mentioned shortcomings (missing beam search) by implementing a custom beam search algorithm.