Google has published an academic paper wherein six of the company’s best AI researchers and one from the University of Toronto team up to create a template of sorts for a machine learning model that could potentially be used to create an all-in-one, universal AI. The powerful all-around model described works by incorporating multiple convolutional layers, sparsely-gated layers, and an attention mechanism. A convolutional layer, to put it simply, is not unlike a single node in the human brain, while a sparsely-gated layer is an open-ended neural network node that can decide what in-network resources to use for a given task. With these resources combined, the researchers created an AI that was able to perform consistently well across many different tasks.
The model is able to essentially stack layers like modules. While a normal AI would be bogged down and possibly end up confused by this process, the way that the layers combine in this model, with the attention layer to help dictate what nodes should be active during a given task, was able to perform slightly better at all tasks as more skills were learned. Essentially, the AI was showing signs of skill synthesis, a very human learning trait. On top of skill synthesis, the AI model created to demonstrate this blueprint was able to perform large, complex tasks that required the participation of multiple layers with barely any degradation of performance. Essentially, this concept could be a brand new application of the concepts of sequential learning and artificial general intelligence that’s more human in the way it learns than anything that came before it.
This model, depending on how well researchers optimize it and how popular it becomes, could easily become the basis of an entirely new breed of AI. Systems and models built on this blueprint could, in theory, stack an infinite number of layers, so long as the entity building and maintaining the model had the resources for it. Such a model would be able to learn just about anything, and even go through the complex series of calculations required to imitate emotion and self-awareness, though machines truly adopting these qualities is a very long time away, if it’s going to happen at all.