Developers responsible for the machine learning found in Google’s Gboard application have created a new machine learning model that may be able to discern even some of the worst handwriting around, based on a recently reported explanation from the company.
The AI-driven keyboard mode has improved substantially since its initial launch but the latest changes may be the biggest yet. New revisions to the machine learning behind the handwriting feature enabled by advances in AI have now resulted in new model architectures and training methodologies that allow an improvement of 20- to 40-percent over the previous iteration.
The latest method is described in some detail in a new paper published by the company and seems exceptionally complex at first glance but may actually be much more intuitive. It’s based on the introduction of recognition for touch points, Bézier curves, and recurrent neural networks (RNN) — specifically quasi-recurrent neural networks (QRNN).
What that means is that the system no longer depends entirely on “hand-designed heuristics” that simply separate strokes into separate characters. Instead, it starts by noting touch points that represent the start point of a given stroke. Those are time-stamped to assist in separating which character they’re meant to be a part of but the system also calculates and factors those points into computer-readable Bézier curves — essentially cubed off sections that allow more accuracy in reading the curves of a stroke.
The QRNN acts by alternate between convolutional and recurrent layers, reading in the curves and touch points before decoding the handwritten text into letters.
Summarily, the AI reads each character as its being written in a way that’s more similar to how a person might watch the letters being written out — the flow of the handwriting, so to speak. That allows for a better guess based on a more natural set of parameters that are easily identified by the system, with each individual curve being compared against a dataset to produce the resulting probabilities, default output, and suggestions.
Lightweight for mobile
Aside from improving the accuracy of the handwriting mode in Gboard, the developers behind the tool took things further by also ensuring it’s still lightweight enough for the mobile environment. To maintain accuracy and speed things up, the team trained its AI in TensorFlow before shifting everything to TensorFlow Lite models. That allowed for a decrease in the overall download size of the tool but also for the training module to be reduced in terms of “bytes per weight” down by around 75-percent.
That’s an important aspect of the implementation since a keyboard is, first and foremost, a text input and communication tool. When used in a messaging or note-taking app, in handwriting mode, it needs to keep up with the user’s writing speed as closely as possible.
In the wild?
For the time being, the new feature is only going to apply to latin-script languages such as English and those that use similar sets of characters but that shouldn’t remain the case for long. Although there’s no specific timeframe laid out yet, Google says that its team is already working to ensure that it will work with Gboard’s more than 100 presently supported handwritten languages. That means it should appear soon enough for most users.
