Report

Teaser, summary, work performed and final results

Periodic Reporting for period 3 - MOTO (the embodied reMOte Tower)

Teaser

Human Performance advances in Air Traffic Management (ATM) have been traditionally focused on two main senses: sight and hearing. One of the most emphasized findings in the HP Assessment for Remote Towers [SESAR, 2013] is the low quality of the “real world view”. This...

Summary

Human Performance advances in Air Traffic Management (ATM) have been traditionally focused on two main senses: sight and hearing. One of the most emphasized findings in the HP Assessment for Remote Towers [SESAR, 2013] is the low quality of the “real world view”. This finding lead to increased recommendations and efforts towards the improvement of video resolution. However, there is an unexplored potential of perceptual improvement for ATCOs performance in Remote Tower Operations (RTO), namely in the use of the other human senses.
In this situation, there are two major research opportunities:
• The first one is to consider the role of all the human senses for tower operations. The approach of Embodied Cognition [Anderson, 2003; Wilson, 2002] could be applied to achieve a full understanding on the use of all senses for controllers, besides the visual one.
• A second related opportunity is to build new human-system interaction concepts on the understanding of embodied aspects of ATM Human Performance. The final goal is to enhance human performance by augmenting multisensory stimuli – including but not limited to the already overloaded visual channel - without increasing the ATCO workload.
The overall objective of the project is to identify the key multimodal stimuli required on RTO to enhance the sense of Presence experienced by ATCOs.
The state-of-the-art of cognitive neuroscience has shown that for humans to really believe to be there – and act accordingly - the integration of multisensory information is more important than the high fidelity of the visual channel in isolation. Multimodal realistic stimuli are interpreted by the brain as generating an illusory feeling to be physically present in the perceived environment.
MOTO project detailed objectives are the following:
• Objective 1: assessment of the role of Embodied Cognition in control tower operations (i.e. the role of multimodal (i.e. multisensory, beyond visual) acquisition of information in current control tower operations).
• Objective 2: definition of user requirements for a multimodal Remote Tower, to reconstruct multimodal perception in a remote tower simulation platform. This objective includes the development of augmented multimodal interfaces, using at least the visual channel and spatialisation of sound.
• Objective 3: definition of brain-physiological indexes, customized for Remote Tower operations, to monitor aspects of Human Performance like: workload, situation awareness, drop of attention.
• Objective 4: Validate the above results in realistic ATM operational conditions through simulation facilities. The validation will target two scenarios:
a. Compare performance benefits between a baseline Remote Tower and the Augmented Remote Tower.
b. Assess the performance benefits of the Augmented Remote Tower.
Current target scenarios include: high traffic airport controlling in parallel a low traffic airport, more than two small airports in parallel, controllers as supervisors of highly automated airports.

Work performed

The work performed in the third reporting period mainly addressed the preparation, execution data analysis and reporting of the first MOTO validation.
The first validation was performed in SCN Laboratory in Rome. The main aim was to understand the impact of different sensory inputs on Air Traffic Controller’s performance in Remote Tower Operations (RTO).
It was also assessed if the selected technologies can reproduce a realistic experience of visual, audio and vibration signals in ROT, in order to identify user requirements for realistic multimodal remote tower platforms.
Finally it was investigate the development of the embodiment index, correlating the physiological parameters of each participant with the embodied experience in the simulated virtual environment.
Fifteen professional Air Traffic Controllers (ATCOs) participated in the first validation experiment. The experiment was performed in an interactive virtual reality environment reproducing a Control Tower using the HTC Vive Head Mounted Display platform.
The participants were asked to manage different realistic Tower Air Traffic Control operational scenarios. These scenarios were rendered in different complexity levels (easy vs hard) and in four sensory modalities conditions ( i) visual; ii) visual and auditory; iii) visual and haptic; iv) visual, auditory and haptic).
To investigate human performance, sense of presence/immersion, situation awareness and cognitive workload a combination of subjective and objective measures were used: 1. Neurophysiological signals (Electroencephalogram, Electrocardiogram, Galvanic skin response); 2. Subjective questionnaires; 3. Subject Matter Expert evaluation.

Overall, the main outcomes of the first validation experiment are summarized in the following points:
• Participants experienced high sense of presence in the VR immersive RTO during the scenarios execution.
• Subjective and neurophysiological indicators collected suggested that performance of ATCOs could be enhanced by integrating visual channel with one sensory modality at a time (either audio or haptic).
• Workload analysis showed that by integrating only Audio channel to the Video one (i.e. VA condition) induced a decreasing in experienced workload with respect to the other conditions.
• By mixing and providing ATCOs with visual and both Audio and Vibrotactile stimuli, it seems to induce a degradation of performance, maybe due to distracting stimuli contribution that is higher than their informative content.
• The Embodiment Index has been developed as an objective measure of the sense of presence and was estimated by the modulations of the GSR responses induced by an emotionally salient virtual context in RTO environment.
• The GSR results from the whole experimental activity and those obtained from the neurophysiological EEG suggest, consistently, that in the VA condition ATCOs sense of presence is improved, along with performance and workload improvement. However, no statistically significant correlation has been identified during the analysis.

Results from the first validation activity allowed to identify a list of user requirements for implementing more immersive multimodal remote tower platforms. A detailed description of each requirement is reported in D4.2.

During the third reporting period it was also carried out the two pilot studies in preparation of the second MOTO validation.

Final results

The main expected benefits coming from the project to Remote Tower Operations are the following ones:
1. Enhancement of the Sense of presence via multimodal solutions. The enhanced Sense of presence is a key requirement to preserve human performance as in real operations, with the same attention levels, similar decision-making processes, in order to facilitate the transfer of real-world knowledge and skills into the RTO. Multimodal solutions will be tested in MOTO second validation activity.

2. The development brain-physiological indexes, customized for Remote Tower operations. MOTO will develop a non-intrusive index based on kinematics, physiological and brain cortical activity, to monitor the condition of controllers and detect any loss of situation awareness due to lack of realism (or other technical problems) in the Remote Tower platform. The fine tuning of the index will be performed in the MOTO second validation activity.

Website & more info

More info: http://www.moto-project.eu/.