Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - ACHIEVE (AdvanCed Hardware/Software Components for Integrated/Embedded Vision SystEms)

Teaser

Applications like assisted and autonomous driving, unsupervised surveillance or robot vision, mandate real-time interpretation of the scene. This requires the incorporation of a higher level of intelligence at sensor level in order to extract the relevant information. The goal...

Summary

Applications like assisted and autonomous driving, unsupervised surveillance or robot vision, mandate real-time interpretation of the scene. This requires the incorporation of a higher level of intelligence at sensor level in order to extract the relevant information. The goal will be to analyse the visual stimulus and to elaborate an adequate representation of the scene right at the sensor plane. This has very positive consequences for the power efficiency of the system. Concepts like computational image sensors, silicon retinas and vision sensors are being taken into consideration again, because of a shift in the target of the visual information. Until recently, the end user has been the human visual system, nowadays the addressee of this information is a computer.
Current trends in object recognition and classification rely on representation learning. Provided with a flexible internal structure and enough computing power, modern machine-learning systems has superseded the extraction of handcrafted features in favour of automatically discovering the most appropriate internal representation. This deep learning approach has revolutionized object recognition by dramatically reducing the error rate. The challenge today is to convey these processing capabilities to compact, lightweight and power-aware embedded vision systems and vision systems-on-a-chip. Moreover, in application scenarios like autonomous surveillance or intelligent transportation systems, these embedded vision systems will also be networked. A centralized processing is unpractical, it is necessary to develop a distributed system in which smart devices cooperate towards a collective goal.
The approach to take these challenges needs to be multidisciplinary. Efficient analysis and multilevel optimization techniques are required. Experts in the design of smart image sensors, vision systems-on-a-chip, energy-efficient embedded vision systems, parallel processing architectures, fast and efficient feature extraction and learning and computer vision algorithms need to converge on common ground. ACHIEVE-ITN aims at training a new generation of scientists through a research programme on highly integrated HW-SW components for the implementation of ultra-efficient embedded vision systems as the basis for innovative distributed vision applications.

Work performed

The work carried out in this first half of the project can be divided in two phases. In the first phase (Oct. 2017-Sep. 2018) the effort has been concentrated in setting up ACHIEVE’s network and providing the appropriate setup for the development of the research and training programme. At the beginning, the activities were conducted to build the network structure:
• constituting the different governing boards and supervisory structure;
• publishing the Project Handbook;
• issuing several compliance reports on EU policies;
• consolidating agreements for the co-financing of project management expenses and network-wide training activities;
• identifying the main strategies and means for project promotion, communication and dissemination.
On Nov. 8-9, 2017, a kick-off meeting was held at the facilities of CSIC in Brussels. The meeting was attended by representatives of all the beneficiaries and the majority of the partner organizations. During the talks we designed the recruiting campaign and consolidated the calendar of training activities. A general recruitment call was issued, and total of 303 applications were evaluated with homologous procedures. 9 ESRs were hired.
In the second phase (Oct. 2018-Sep. 2019) we have started the research and training programme. Several network-wide training activities have been realized during this year:
• seminars on open science principles and on ethics in scientific research and publication;
• a course on feature learning on embedded systems was held in connection with the celebration of WASC 2019 at INSA-Rennes;
• a PhD forum was celebrated at ICDSC 2019 in Trento.
In parallel with the training program, they have started their Individual Research Plans. Briefly on the scientific progress of this year, we have advanced on:
• the design of CMOS-compatible sensing structures that are able to capture both 2D and 3D information form the scene;
• A/D conversion methods for image sensors, and started the design of signal processing blocks for time-to-digital conversion;
• the design of digital signal processing blocks that exploit the inherent parallelism in image data;
• an heterogeneous architecture for an embedded vision system that combines flexibility of FPGAs and robustness of closed-architecture processing units;
• the analysis the requirements on metadata to be employed in cooperative vision;
• the analysis of obstacle detection algorithms for the navigation of autonomous transport and robotic platforms;
• a suite of video analytic tools that will be required for the smart monitoring of human activities.
As already mentioned, this is the outcome of the first-year work of the ESRs. The results are still preliminary in some lines, and channels for interaction between lines running in parallel have been just formally established. We expect to initiate now a round of increasingly coordinated activities.

Final results

The main scientific objective of the research programme is the development of a distributed vision platform composed of networked, smart and efficient embedded vision systems. This platform will be the basic infrastructure for several application scenarios that demand cooperative vision based on the in-node processing of the visual stimulus and extraction of relevant information. This de-centralized scheme renders the system scalable, easily deployable and resilient to partial failure.
Expected advances cover different levels of the system hierarchy. At sensor chip level, we will pursue integrated sensing and processing chips that combine image capture with on-chip acceleration of feature extraction/learning with a limited number of resources and under a restricted power budget. At the level of the embedded system, we will concentrate on compact, efficient and reconfigurable hardware for local processing of visual information combined with agile transmission of metadata and a careful power management.
Beyond scientific impact, this Project aims at innovative products and services. ACHIEVE-ITN participants will enjoy a combination of research excellence and exposure to industry needs and procedures, through the participation of industrial beneficiaries and partners: Imasenic, FLIR, Kovilta, Prefixa and Nvidia. This will provide them with a clear idea of the possibilities of technology transference and the benefits derived from the conversion of their own research results into product and services of high added value.
On the societal side, legal and moral concerns on uninterruptedly surveyed public and private spaces need to be compensated by in-sensor disaggregation of sensitive data. Embedded system and distributed vision enable a level of artificial intelligence that can no doubt interact very profoundly with our experience as free citizens. ACHIEVE-ITN participants will be trained in these social, legal and ethical implications and will be made aware of the dangers of a technological dystopia.

Website & more info

More info: http://www.achieve-itn.eu/.