DeepMind, a Google-owned company focused on developing artificial intelligence-based technologies, has outlined how the British firm developed two kinds of AI that can plan the future using the capacity for imagination. Called imagination-augmented agents, the new types of AI use what DeepMind calls an “imagination encoder”, which is essentially a neural network that helps inform the agents’ decision-making process by learning to gather useful data while rejecting irrelevant information.
According to the papers in which DeepMind described the imagination-based planning approaches, key features of the new family of AI include the ability to interpret internal simulations in order to employ models that, albeit imperfect, still understand the dynamics of the environment surrounding them. The new types of AI can also adjust the imagined courses of action in order to address the problem. Additionally, the imagination encoder improves the AI’s efficiency as it extracts more data from imagination that does not necessarily lead to a high reward. Another feature of the new AI family is the ability to create plans using a variety of strategies that include opting to carry on with the imagined course of action or choosing to start all over again from the beginning. The planning strategies are further enhanced with the addition of various imagination models that have different levels of accuracy so that the agents’ approaches are not limited to only a few trajectories.
To test the new approaches, DeepMind used games that demand forward planning and reasoning as the ideal ecosystem in which the agents’ capacity for imagination-based planning have been evaluated. For comparison purposes, DeepMind pitted the imagination-augmented agents against imagination-less baselines and the results speak for themselves: the former surpassed the latter. The agents also worked to try the different strategies internally before carrying them out in real-world scenarios, as DeepMind generated each task level in such a way that the agents may only perform a strategy once. The agents performed the tasks even better after DeepMind added an element of “manager” to them to assist in making a plan. The company observed that an agent learns to make a compromise when various models of an environment are presented to it with a varying degree of quality and cost-benefit. The papers seek to address questions about how the agents can deal with imperfect models and adapt a strategy to the status quo. An update on DeepMind’s new endeavor is likely to follow in the near future.