FASTDEFORM

Real-time understanding of dexterous deformable object manipulation with bio-inspired hybrid hardware architectures

 Partecipanti

Mappa

 Word cloud

Esplora la "nuvola delle parole (Word Cloud) per avere un'idea di massima del progetto.

 Obiettivo del progetto (Objective)

'The ability to perceive and understand dexterous object manipulation from visual information, irrespective of an object’s physical characteristics, is critical for next generation robotic systems. It enables fluent interaction outside the highly constrained environments of today’s applications and allows robots to enter domestic settings and natural environments. Many approaches have been proposed to this end, but to enable real-time performance, all of them need to severely restrict model detail, degrees of freedom of articulated objects, and/or deformation complexity. The real-time aspect is critical since the perception of deformable objects under manipulation seems to be a dynamic process, in which specific actions may be efficiently used as active exploration primitives towards model abstraction. This requires closed-loop perception-action cycles. In this project, the candidate will develop a theoretical communication framework that removes a critical obstacle in current systems: the sensory/semantic communication bottleneck. In accordance with biological vision systems, the feedback and feedforward communication between dense low level sensory and compact high level semantic representations will be shaped using visual attention methods. An internal model representation will be developed that supports varying degrees of invariance to an object’s physical characteristics, and the action state space will be reduced by considering objects together with their manipulators. The candidate will also realize the communication framework as a real-time hybrid architecture by combining conventional (CPUs) with graphics processing units (GPUs). By providing multiple relevant contributions across the spectrum of the FP7 objectives in terms of its potential to advance robotic manufacturing, medical image processing, and computing paradigms, this project will enable the candidate to maintain and enhance his position at the forefront of advances in this field.'

Introduzione (Teaser)

The ability to process and utilise strictly visual information during object manipulation regardless of an object's physical traits is critical to next-generation robotics. New technology has removed previous barriers and outperformed competitors.

Descrizione progetto (Article)

Many approaches have been proposed, but achieving real-time processing of closed-loop perception-action cycles has necessitated severely simplified system models. This poses a restriction in terms of identifying sensory information of meaning to the task (sensory-semantic communication).

Scientists developed novel methods to build and update a detailed 3D scene representation based on sensory information extracted from the scene in real time. With EU support of the project FASTDEFORM, researchers delivered a pioneering system that surpassed performance of state-of-the-art methods.

FASTDEFORM exploited the massive parallelism of graphics processing units, programmable logic chips that perform rapid computations primarily for the purpose of rendering images, animations or video. Concepts based on visual attention enable focusing on scene items of interest at a level of abstraction supported by both real-time sensory and prior information.

Researchers achieved significant enhancements in the speed, robustness and accuracy in perceiving and understanding a complex dynamic scene. This was accomplished through real-time integration of visual simulation and visual perception on the same hardware.

The highly robust and real-time system operation was demonstrated in three different tasks. These covered manipulation of a large number of rigid objects, manipulation of a complex foldable cardboard brochure and vision-based robot control using feedback from a vision sensor. More information on each is available http://www.youtube.com/user/karlpauwels (online).

FASTDEFORM technology provides robotic systems with the ability to perceive and understand complex manipulations of non-rigid and dynamically changing objects. This enables them to move out of the realm of highly constrained industrial settings to novel applications in manufacturing, medicine and domestic environments. It has the potential to usher in a new era of devices exploiting human abilities in the support of human endeavours.