#	Pagina
attuale pagina	/open-h2020/projects/205934/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - INSPEX (Integrated Smart Spatial Exploration System)

Teaser

Summary

Obstacle avoidance systems for automotive is an active research domain. They combine multiple sensing technologies (LiDAR, radar, IR and visual) to detect obstacles across all lighting and weather conditions. These systems are large, heavy, not fully integrated in a unique system, power hungry and require large computational capabilities. They are indeed limited to high-end vehicles. Moreover, GPS and/or Inertial Measurement Unit (IMU) based traditional navigation systems are not enough to ensure safe navigation of users because unmapped obstacles cannot be avoided.
INSPEX objective is to make obstacle detection capabilities currently implemented in high-end vehicles available as a portable multi-sensor, miniaturized, low power device. The INSPEX spatial exploration system will be used for 3D real-time location and warning of static and mobile obstacles under various environmental conditions in indoor and outdoor. Potential applications range from human navigation in reduced visibility (e.g. for firefighters), small robot/drone obstacle avoidance systems to navigation for the visually/mobility impaired.
The INSPEX system will be integrated in a regular white cane for Visually Impaired and Blind (VIB) and provide 3D spatial audio feedback on obstacle location to its user. This use-case is considered highly demanding in terms of miniaturization, integration challenges, power efficiency and needs for communication with the smart environment. The choice as primary use-case for a smart white cane may have societal impacts. Actually, according to WHO statistics, 285 million people are VIB world-wide and this number is expected to double by 2040, due to aging and health diseases. Moreover, navigating in a city is far from safe for VIB people and sooner or later, they sustain injuries (especially on head and chest) due to unexpected obstacles. Therefore, electronic white canes, able to detect obstacles on the whole person height should improve VIB confidence in their mobility capabilities.

Work performed

The project applied a bottom-up approach, starting from potential users\' interviews to collect their needs. Use-cases both from user and system perspectives have been derived together with system requirements. Then, an integration strategy has been issued to explore different integration solutions and their fit with user\'s needs and requirements.

The INSPEX architecture (HW&SW) has been defined. The system is made of three parts:
- a mobile detection device (MDD) that embeds range and environmental sensors, an IMU and computational capabilities to fuse data and build a model of the user\'s surroundings. This model is then analysed to transmit information regarding potential harmful obstacles that enter the user\'s safety zone
- an smartphone apps that generates 3D spatial sound from information received from the MDD and from the head orientation
- a extra-auricular headset that embeds an IMU to send head orientation to the smartphone and provide an accurate 3D audio location of the harmful obstacles
In parallel to this architecture activity, partners brought state-of-the-art range sensors (LiDAR, UWB radar, MEMS ultrasound) to the project. The sensors were first characterized, then they have been miniaturized with dramatic effort to reduce their power consumption . A first version of these optimized sensors has been delivered for their integration in the first integrated prototype (the latter is out of the current reporting period). The MDD as well as the different optimized range sensors have been designed and fabricated.

Legal and Ethical issues raised by the development of such a system have been considered from the beginning of the project, together with the legal rules the GRDP imposes.
An early prototype has been built during the first year of the project. It integrates off-the-shelf range sensors, IMU and demo-kit computing platform. The main goal for this early prototype was to validate the basic concepts the INSPEX system is rooted in, and develop the INSPEX firmware in parallel to its hardware. The architecture of this early prototype is as close as possible to the architecture of the INSPEX integrated prototype, especially regarding the computing platform and data exchanges. Even if this early prototype does not offer the full spectrum of capabilities of the INSPEX system, it has been used during validation tests in laboratory conditions.

A survey of several formal tools has been done to evaluate their use in the context of INSPEX firmware development. A bottom-up approach has been taken to verify key parts of the firmware and improve its reliability. A top-down approach has been taken to model the power management strategy. Analysis of Off-The-Shelf components has identified a \'black hole\' state in the Bluetooth power system, which must be avoided in the implementation of INSPEX.
The INSPEX system is part of the smart environment. A test environment has been setup to enable evaluation of the use of BLE beacons to provide context information to the INSPEX solution. Deployment and customization of beacon tags and the development of an Android smartphone application for scanning, connecting and interacting with the beacon infrastructure have been performed. A demonstration of this was done at CEA site during a F2F meeting.
Details on these activities will be reported in the second reporting period.

Final results

INSPEX progress beyond the state of the art includes:
- decrease of power consumption, size and weight of the range sensors integrated in the MDD (ratio 2 to 7 depending on the characteristic and on the sensor)(several publications, esp. SSI\'18, , ECTC\'18, one patent filled)
- system modular design, esp. of the MDD (HW&SW)(Best presentation award DAC\'2018 IP-Track)
- construction of the 3D surroundings model in real-time on a microcontroller (the 2D model was available at the beginning of the project)
- definition of a sensor model for large field-of-view sensors used when the model is computed (patent application)
- modular firmware definition for the data acquisition subsystem (software author right protected by APP French application)
- study of the system reliability that lead to some technical choice in the design and fabrication of the different pieces, especially the mobile detection device
- 3D audio feedback taking into account the user\'s head orientation to improve his/her experience and ease the true localisation of obstacles (beta version available at the end of the first reporting period)
- legal and ethical related issues part of the outcomes of the project (several publications in this field)

Note that INSPEX follows two tracks in its development and in routes for future exploitation. Basically, the white cane application, even if of high societal impact, is a niche market. Thus, the range sensors miniaturized in the course of the project must meet other application domains (e.g. consumer domain): they are developed as independent modules with their own value in the exploitation strategy. The second track is related to the whole INSPEX system, with primary application for the VIB Community.

During the second reporting period, the system will be even more tightly integrated, with a second version of the miniaturized sensors delivered at month 24 (i.e. Dec. 2018). It will be validated in real-life context with VIB volunteers during the second semester of 2019. Moreover, advanced functionalities regarding 3D audio feedback