Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - Dreams4Cars (Dream-like simulation abilities for automated cars)

Teaser

A human being learns to drive in tens of hours. On obtaining a driving license, a driver initially lacks experience, and is consequently exposed to higher risks than senior drivers. But, while driving, humans improve their behavioural-control skills; following unexpected...

Summary

A human being learns to drive in tens of hours. On obtaining a driving license, a driver initially lacks experience, and is consequently exposed to higher risks than senior drivers. But, while driving, humans improve their behavioural-control skills; following unexpected events they can predict potential risks and learn better behaviours. This process is so effective that the average fatality rate of human driven cars is of the order of 1 every 100 million miles.

Autonomous vehicles generally lack such learning abilities, improving only via the intervention of human designers, and requiring long real or virtual testing to find situations where the car is unsafe. The causes of incorrect behaviours are then diagnosed and the control software updated/re-tested. So far, and despite the resources spent in the last decade on many initiatives, no autonomous vehicle has yet proved to be even close to the human accident rate (and they should be better than human for mass introduction).

Is it possible to engineer cars that can learn by re-elaborating their own driving experience? More generally, is it possible to develop a robot that can re-use experience gained during operation in order to imagine potential threats and evolve better behavioural-control strategies?

Dreams4Cars is about creating a system with this capability. It aims to benefit the development of automated vehicles in the first instance, but could also have an impact in robotics and artificial cognitive systems in general.

The theoretical framework in which Dreams4Cars falls is the Simulation Hypothesis of Cognition, which describes various forms of mental imagery as a process of simulation of actions and perceptions. The human dreaming state is thus the underlying inspiration for Dreams4Cars, such that an agent, while driving (the ‘waking state’), records events and situations and then, offline (during the ‘dreaming state’), creates fictional situations in order to discover potential threats and train its own sensorimotor system.

The concrete objectives of the project are thus the following:
1) Implement an artificial driver with a sensorimotor architecture capable of learning models of the world and recording events for future dreams and, also, capable of changing its own configuration to implement new/improved behaviours learned from the dreams.
2) Implement imagery mechanisms that produce novel situations via episodic and embodied simulations, to optimise behaviours and control abilities offline, and communicate adaptation for online use to the artificial drivers.
3) Demonstrate the effectiveness of this technology by evolving one driving agent with cycles of activity and simulation on research-grade vehicles.
4) Port the resulting agent to a real production vehicle and demonstrate the achievement of TRL 6, as well as an increase in the level of abilities compared to a benchmark agent which was developed in the FP7 AdaptIVe project.

Work performed

Objective 1. Guidance from a number of theoretical and experimental brain studies have been used to define the agent architecture, implemented as shown in the picture. Among these, Cisek’s affordance competition hypothesis had a prominent role, suggesting the use of 4 loops: 1) The dorsal stream implements parallel action priming encoding affordances in the form of activity peaks in the “artificial motor cortex” of the agent (the output tensor of a deep neural network labelled with “c”). Following Damasio, the stream is implemented with convergent (from a to b) and divergent zones (from b to c) that force categorization of sensory data in b. A branch running backwards from b to a is also implemented. 2) The action-selection loop follows works from Gurney and others, and implements robust action selection and adaptive behaviours. 3) The frontal loop follows ideas from Pezzullo and others, and implements biases in action-selection such that the agent is able to suppress selection of immediately higher-rewarding choices to favour of longer-term advantages. 4) The cerebellar loop follows work by Anderson and others, and implements the learning of body-level forward/inverse models. It complements the dorsal stream categorization in “b” to provide the elements upon which dream-like imagery is based.

Objective 2. Following works by Svensson and others, two forms of simulation have been identified: “episodic simulations”, that are more abstract and related to cortical-subcortical processes, and “embodied simulations”, which are more detailed and related to the cerebellar loop. Two mechanisms have been developed for episodic simulations. One uses the convergent-divergent structure of the dorsal stream to create sensor imagery in “a” based on recombination of precepts in “b”. The second episodic simulation mechanism is based on genetic algorithms for recombination of elementary items and events into fictitious situations. Concerning embodied simulation, various methods for the learning of forward models have been studied, amongst which is a neural network approach that follows the architecture of the cerebellar adaptive filters. Learning occurs in all the 4 loops and various approaches to the training of the loops have been developed. An environment to host simulations has been developed based on the open access driving simulations tool “OpenDS”.

Objectives 3 and 4. Preparatory work has been carried out, which includes the preparation of the test vehicles, the arrangement of the test sites and the plans for the experimental activity.

Final results

Dreams4Cars impact is targeted at two interrelated domains: robotics, in general, and automated vehicles, in particular (the principle application domain).

The agent architecture, as described in objective 1, is a brain-like sensorimotor system that implements a number of cognitive processes that (together and individually) constitute an advancement of the state of the art, both by comparison to sense-think-act human-engineered systems as well as in comparison to end-to-end deep learning implementations. The effectiveness of this architecture has already received several corroborations (e.g. see the project video https://youtu.be/-AxNUxXQRUM).

Furthermore, a number of episodic and embodied simulation mechanisms (objective 2) that allow the agent to learn via self-instantiated offline simulations have also been developed and (in part) validated.

Finally, an open access (OpenDS) simulation environment for promotion of project results and further research (within which the agent’s capabilities can be openly demonstrated) has been developed and will be made freely accessible.

Although the agent has been developed specifically for the task of driving, it has the potential to impact the wider robotics domain as a concrete example of a biologically-inspired cognitive architecture (the architecture may be arbitrarily extended with further brain-inspired loops and functions).

As has been demonstrated to date, the system contributes to the attainment of many robotics abilities listed in the robotics multiannual and strategic research agenda. This includes progress in the level of Adaptability, and intrinsic Cognitive ability. The application of these to the transport domain is expected to contribute to the development of safer and more reliable autonomous vehicles, strengthening the EU industrial base, contributing to the reduction of road accidents and improving mobility services.

Website & more info

More info: http://www.dreams4cars.eu.