Facebook has announced that they have come up with a way to use a convolutional neural network to translate text, and it can do its job about nine times faster than most current solutions in the same vein. The Facebook Artificial Intelligence Research team developed an advanced model that takes advantage of the unique features of convolutional neural networks, allowing those factors to work with translation in much the same way they work in things that convolutional neural networks normally do. The training framework and pre-trained model are also open-source, which means that anybody who wishes to can implement them in their own use cases.
Convolutional neural networks are normally used for less exact tasks, like image processing, and normally don’t fare too well with translation tasks. They’re normally soundly outperformed by recurrent neural networks, which tend to do better with tasks involving rigid rules due to their strict right to left or left to right instructional flow. A convolutional neural network, on the other hand, takes the entirety of a bit of data into account, and works off of that. This would normally result in translation tasks being slow and somewhat inaccurate, but the FAIR team’s model has been trimmed down for computational efficiency, and has been taught to analyze phrases by chaining together literal meaning and context, then using the combined information to synthesize a translation. Essentially, the neural network does multiple passes, focusing on the phrase differently each time. Once the English words are loaded up, they can be computed simultaneously and put in the correct order, with tweaks for grammatical correction, almost instantly. The concept is a bit on the complicated side, but an animated infographic was provided to help those outside of the machine learning field to understand it.
The implications of this development reach far outside of the realm of translations. The multi-pass approach employed here empowers CNNs to do things that they had trouble with before in entirely new ways, and can be used by neural networks to simultaneously compute multiple takes on a conversation, as well as multiple aspects. This means that things like responding more naturally, understanding complex phrases, following the flow of a conversation, and following up on a previous question or conversation point can all benefit from this model.