#	Pagina
attuale pagina	/open-h2020/projects/203543/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - GOAL-Robots (Goal-based Open-ended Autonomous Learning Robots)

Teaser

Summary

\"What is the problem/issue being addressed?

Imagine your friend asks you to tidy up her kitchen which is full of furniture and objects. Now imagine that your friend described through a photo how the kitchen should look like after being tidy up. This task would be boring for you, but you would nevertheless be able to carry it out with ease. Indeed, when you were a child you played with all sort of objects and so, driven by curiosity, you learn many flexible sensorimotor skills to manipulate them at will. Differently from you, current robots are ill-suited for these type of challenges. Indeed, although new architecture and algorithms are developed to control robots, leading them to accomplish tasks in unstructured environments still requires much programming or supervised training. Moreover, in such current robots can become able to solve specific tasks but in comparison to biological agents they have a very limited autonomy and flexibility.

Why the project is important for society? For many applications, future robots should be able to learn how to solve multiple tasks in unstructured environments in an autonomous way. The project aims to build artificial intelligence architectures that should allow robots to self-generate goals and autonomously learn sensorimotor skills to accomplish them, based on the cumulative acquisition, modification, and recombination of previously acquired skills. This will allow robots to accomplish usersâ€™ tasks (e.g., to set a room in a desired state) with little additional learning.

What are the project objectives? The project follows a previous European project called IM-CLeVeR (â€œIntrinsically Motivated Cumulative Learning Versatile Robotsâ€) which investigated how intrinsic motivations (IMs) can support autonomous learning in biological and artificial agents. IMs, related to novelty, surprise, and competence acquisition, are maximally apparent in children at play: driven by IMs, children explore and interact with objects in the environment thus getting to know how they work and to acquire a wide set of sensorimotor skills suitable to manipulate them as desired. GOAL-Robots central idea is that IMs can fully support open-ended learning only if a further critical ingredient is considered: goals. A goal is an internal representation of a state (or set of states, or trajectories of states) of the world that can be internally activated by the agent in the absence of its current perception: when a goal representation is internally activated, it can contribute to move the agentâ€™s attention, behaviour, and learning resources towards the accomplishment of the goal. An example of goal is \"\"I would like to see that glass of water in my hand\"\" and an example of action to accomplish such goal is \"\"I will move my hand and grasp the glass\"\". An example of more complex goal is the one considered above \"\"I would like to tidy up the kitchen after dinner\"\". The project idea is that endowing robots with the capacity of self-generating goals, and using them to learn the skills for their accomplishment, is the key ingredient to have truly autonomous open-ended learning robots.
\"

Work performed

The project concept is pursued through three main objectives: (1) advance our understanding of how goals are formed and underlie skill learning in children; (2) develop innovative computational architectures and algorithms supporting (2a) the self-generation of useful goals based on user/task-independent mechanisms such as intrinsic motivations, and (2b) the use of such goals to efficiently and autonomously build large repertoires of skills; (3) demonstrate the potential of project idea with a series of increasingly challenging demonstrators in which robots autonomously acquire complex skills and use them to solve difficult challenges in real-life scenarios. This far the project has achieved a number of results in relation to these objectives.

(1) The project has carried out a number of experiments with infants with the objective of understanding the mechanisms through which they undergo open-ended learning. We investigated in depth â€œcontingency-based learningâ€, a mechanism at the core of infantsâ€™ learning, for which the consequences of actions, internally represented as goals, motivate the infant to refine the actions leading to them. This mechanism was also studied in depth in computational models shown to be able to learn how to produce effects on own body (e.g., to touch own body parts with a hand).

(2) A second achievement was the development of controllers allowing robots to self-generate goals to guide autonomous open-ended learning. The idea is that a robot is left alone in an environment and by exploring it with already-acquired actions it should self-generate goals. One way to do this is based on the idea to form goals as states that follow the robot\'s actions that cause changes in the environment, or based on prediction errors.

(3) The self-generated goals should then guide the robots to acquire skills accomplishing the goals: these skills are used to pursue goals that become desired in the future, e.g. to tidy up a kitchen. This requires architectures that integrate different functions, all implemented in the project robots: deciding on which goal to invest learning time; integrating the acquired skills; learning forward models; re-using the previously acquired goals and skills to accomplish the goals of the robot\'s users, e.g. based on planning.

(4) The project is also showing how the devised systems work in real robots. In particular, we have developed a robotic scenario to test the ability of robots for open-ended learning and then for re-using the acquired knowledge to accomplish externally assigned tasks, e.g. to put some objects back into their desired place. The task can have an increasing complexity, in particular the task can require to simply work on a table plane, or to also move objects on a shelf (which implies grasping the objects), or to configure the objects in a given relative state (e.g., to form a pile).

Final results

The project main expected result is the development of a new paradigm to build open-ended learning robots. This paradigm rests upon two key insights. First, to exhibit an autonomous open-ended learning process, robots should be able to self-generate goals, and hence tasks to practice. Second, new learning algorithms can leverage self-generated goals to dramatically accelerate skill learning. The new paradigm will allow robots to acquire a large repertoire of flexible skills in conditions unforeseeable at design time with little human intervention, and then to exploit these skills to efficiently solve new user-defined tasks with no/little additional learning. The project will advance our understanding of the fundamental principles of open-ended learning in autonomous robotics by producing robots that can autonomously accumulate complex skills and knowledge in a truly open-ended way. This innovation will be essential in the design of future service robots addressing pressing societal needs thanks to its wide applicability. Examples of applications: tidy up indoor and outdoor environments; performance of tasks useful for humans in unstructured environments; entertainment/game activities.