Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - SENSORIANCE (SEnsorial awareNess System fOR obstacle detectIon And collisioN avoidanCE)

Teaser

New rulings are being published enabling landing operations with Enhanced flight vision system (EFVS) under reduced visibility conditions, such as with no visible references like the approach, the threshold, or the runway edge lighting systems. Recently, for example, the US...

Summary

New rulings are being published enabling landing operations with Enhanced flight vision system (EFVS) under reduced visibility conditions, such as with no visible references like the approach, the threshold, or the runway edge lighting systems. Recently, for example, the US Federal Aviation Administration (FAA) enabled trained pilots in the use of EFVS to continue descending 100 feet below the Touchdown Zone Elevation (TDZE), and even perform a full landing procedure with no natural vision under specific conditions. This is leading to new advanced EFVS that meet the requirements to adapt to these new regulations.

One of the main challenges faced by EFVS is the need for systems which are dependable, affordable and available in all-weather operations. The introduction of the latest advances in sensing technology and computer vision algorithms can allow to widen the number of market applications and reduce the price of the final solution for new generation EFVS.

Under this framework, the main goal of SENSORIANCE project is to develop a cost-effective system to provide combines information from various sources, including a compact camera –consisting of a cost effective, versatile, high-performance and highly-reliable optical system— and external sensors, distributed in the form of LRU modules. This information is then processed by a computer vision and image processing module to infer useful knowledge and event-reaction capabilities, aiding pilots during all phases of flight (taxi, takeoff, cruise, and landing).

SENSORIANCE system will provide a complete enhance and synthesized images suite and it will allow to enhance the field of view, to reduce the risk of occurrence of an impact, to Enable air transport system optimization, to ensure compliance with safety standards and regulations, to use ROHs components, requiring low investment in devices (COTs).

Based on these general objectives, the technical objectives of SENSORIANCE are the following:
• To develop an affordable uncooled VIS/IR sensor.
• To develop software components that will consist of several modules to perform the following tasks: manage user input and console commands, present feedback messages to the user, control hardware modules, setup and maintenance automation and computer vision functionality.
• To develop a precise mechanical assembly of lenses that will meet the different focal lengths required by the system. Image enhancement capabilities with a good compromise on cost, performance and weight will be introduced.
• To design, develop and manufacture of mechanical casing and electrical circuits under RTCA/DO-160G for the TRL5 prototype to comply with airborne hardware requirements.
• To develop different modules for image acquisition, image compression and communications via the standard Ethernet or ARINC818 interfaces.
• To design, create test environments for visualization and simulation and perform the different electrical, mechanical and environmental tests specified in the RTCA/DO-160G.
• To design a system with a Mean Time Between Failure (MTBF) for the LRUs of 40.000 h.
• To provide a DRI (Data Requiremente Item) that contain a qualitative assessment of the System and/or LRU BIT capability.

Work performed

As part of the tasks necessary to define the technical requirements of SENSORIANCE, a study has been carried out, first of all, for the selection of optimal detectors spectral bands for compliance with the primary requirement of the project (the improvement of visibility in adverse weather conditions) and secondly, for the evaluation of existing technologies within each type of detector. The shorter wavelengths, makes the most suitable detectors to enhance the visualization of a scene under adverse weather conditions the sensitive to the infrared region of the electromagnetic spectrum, specially the LWIR band detectors where water vapour has a low absorption window.
In order to assure the accomplishment of the requirements for the whole project, different detectors for each one of the bands of the electromagnetic spectrum have been studied and evaluated for use in SENSORIANCE: silicon based detector for the visible and near infrared bands and uncooled LWIR detectors.

To accomplish the mechanical requirements, several options according to the fabrication process and materials selection have been studied. Nowadays there are several fabrication technologies such as Additive Fabrication (AF) and Subtractive Fabrication (SF). The main difference is how the piece is obtained: in additive fabrication (AF) the piece is formed by several thin sheets of material in micrometric scale, which are combined into the final shape, whereas in subtractive fabrication (SF) the piece is formed by removing material from a raw piece until the final shape is obtained. During development, the system’s parts might be susceptible of being manufactured by means of either AF and/or SF. The results would then be compared and studied in order to be able to choose the best option.

Hardware and software modules of SENSORIANCE have been defined. The pipeline inside SENSORIANCE follows a sequential processing order, where the environment is sensed via the VIS, NIR, and LWIR detectors and put together into a coherent video stream by their respective acquisition modules. These video streams are then passed on to the hardware processing modules —featuring an all programmable System on a Chip (SoC) FPGA— which fuses all video streams into one combined, enhanced, and augmented multispectral video output containing all the necessary visual information produced by the system. This final combined video output is then converted back by the hardware output modules to produce a standard signal that can be understood by the aircraft’s output devices (e.g. Heads-up display (HUD) or Heads-down display (HDD)).

Final results

Nowadays, there are a number of commercial EFVS systems in service but many still utilize mid-wave IR sensors (operates in the 1-5 μm band) that are cryogenically cooled by liquid nitrogen refrigeration to very low temperatures to gain the sensitivity to see the minute-temperature differences required to image the world via thermal sensing. The cryo-cooled units are heavy, expensive, require pesky maintenance and involve significant cooldown times before they can operate.

SENSORIANCE aims at providing a solution to these issues while, at the same time, using more
affordable technology than the ones already available in the market by combining information from multispectral IR and visual range sensors. The main advantages of SENSORIANCE will be the following: ITAR free, cost effective, weight reduction, wide field of view, low power consumption, modular approach. Increased MTBF, thermal sensitivity, increase of maximum frame rate and reduction of electromagnetic frequency bands.

The main potential impacts of SENSORANCE are:
• Increased safety levels in low visibility conditions and bad weather conditions.
• Lower operations costs for aircrafts operators. A reduction in weather related delays provides an operator with the potential for substantial savings.
• Reduced fuel burn and environmental footprint (CO2, NOx, noise). The fuel savings is achieved by fewer flight delays, fewer missed approaches, fewer flights forced to an alternative destination, and a decrease or elimination in hold times.
• Increase the competitiveness of the European aeronautical industry by developing new and advanced technologies.

Website & more info

More info: http://sensoriance.com/.