#	Pagina
attuale pagina	/open-h2020/projects/194346/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - DANCE (Dancing in the Dark)

Teaser

Summary

Objective 1 concerns the mapping of visual cortex and auditory cortex with 7T fMRI. We scanned 25 participants on a 7T scanner at the Scannexus facilities at Maastricht University (14 blinds and 11 controls). The objective is to compare cortical thickness measurements between blinds and sighted controls.
Objective 2 focuses on developing creativity and social interaction as a closed loop for congenitally blind. We tested the developed technology in public events involving both blind and normal participants.
Objective 3: to identify effective mappings between physical gestures and sound parameters. This objective led to experiments both in lab setups and in concrete public events, involving the 150 citizens participating to Atlante del gesto_Genova. Scientific publications describe the resulting developments and software systems.
Objective 4: to discover the behavioral and physiological signature of social artistic experience. We developed a series of experiments involving groups of participants (mainly voluntary citizens, i.e. non expert dancers). The phenomenon of emotional and artistic contagion, i.e., the spill-over of emotions into a group of spectators or participants, has been tested in the context of the performance of dance.
Objective 5: to develop a technology platform and provide a series of software libraries. The DANCE technology platform was completed and fully exploited and tested in the DANCE public events, in scientific experiments, and in mobile applications that will be freely available over internet. Finally, a project movement repository was developed for the validation of the platform, and for the experiments. This repository will be publicly available for the scientific community.
Objective 6: to achieve a unique collaboration between performing artists, sound designers, vision and social neuroscientists, computer scientists and engineers. This objectives was fully achieved by intensive work with several artists, resulting in joint papers involving neuroscientists, computer scientists, experts in sonic design.

Work performed

DANCE proposes a Multi-Layered Computational Framework of Expressive Qualities, for which we developed a set of extraction and sonification algorithms working on both high-precision (mocap) and low-precision (mobile sensors) motion capture data for several expressive qualities such as Fragility, Lightness, Fluidity, Impulsivity, and Rigidity.

We developed a series of experiments involving single participants and groups performing contact improvisation.The DANCE technology platform was completed and fully exploited and tested in the DANCE public events, in scientific experiments, and in mobile applications that will be freely available over internet. Finally, a project movement repository was developed for the validation of the platform, and for the experiments. This repository will be publicly available for the scientific community.

The works related to DANCE Project were presented on the most prestigious conferences in human-computer interactions such as ACM International Conference on Multimodal Interaction (ICMI 2017), as well as on very specific events related to dance such as 3rd International Symposium on Movement & Computing (MOCO 2016), Interactive Sonification Workshop (ISon 2016), Sound and Music Computing Conference (SMC 2016). A special issue of the Journal of Multimodal Interfaces (JMUI) dedicated to Interactive Sonification has been organized in collaboration with DANCE project partner KTH, with call for papers in 2017 and publication during 2018. These events are presented in details in the following sections.
The DANCE events were addressed to both experts and wide audience. They include specialized workshops, such as a DANCE workshop at the ISon â€“ Interactive Sonification workshop in December 2016 in Bielefeld, but also public exhibitions, including the artistic project â€œAtlante del gesto_Genovaâ€ in collaboration with the famous choreographer Virgilio Sieni: the project, launched in November 2016, attracted about 150 citizens and an Institute for blind people, and involved these people from January to the end of March 2017.
Among others, in collaboration with the Goethe Institut of Genoa, the contemporary dance performance â€œBlind Sightâ€ by Sagi Gross, devoted to the impact that technological and social developments have on human relationships, was presented at Genoa. The same artist in collaboration with the blind pianist Bert van der Brink proposed the performance titled Embodies at Maastricht Jazz Festival.

The creation of a public online repository including DANCE results available for free download, constitute a solid playground for future developments in the directions individuated by the consortium, in particular towards further developments by the scientific and industrial communities. The project is continuing also beyond its conclusion in different directions: for example, the work on interactive sonification will continue also thanks to a call of the EU ICT STARTS VERTIGO Project for an artistic project based on DANCE.

Final results

1. Scientific and technological research. The first result is a fundamental conceptual framework for the analysis of expressive qualities of movement, modeling an observer of a dance performance. This conceptual framework is made of four layers, ranging from the physical signals that sensors capture to the qualities that movement communicate (e.g., in terms of emotions). The framework aims to provide a conceptual background the development of computational systems can build upon, with a particular reference to systems analyzing a vocabulary of expressive movement qualities, in order to elaborate and to experiment different theoretical paradigms and applicative models of translation of these visual qualities to the auditory modality.
2. Inclusion and rehabilitation. The sonification of the main qualities of movement based on the conceptual frame generates new way of experiencing space and movement even without the use of the sight. The automatic systems (seeing the qualities of the movement and translating them in sounds) enable their users to listen to a choreography or to feel, by hearing, a ballet. This kind of cross-modal mediated experience implying the possibility of conveying a different, deeper awareness of the (affective and social) meaning of movement for people with diminished visual ability and to share this embodied knowledge of the relation between spaces and emotions with other people (with or without loss of visual capacity).
3. Artistic production. The sound, instead of preceding or inspiring movement (as in the classical conception of European choreography), is created by the movement itself (by its qualitative characteristics, its intrinsic affective and relational meanings). In the active collaboration with the choreographer Virgilio Sieni, director of Biennale Dance Festival, we are programming international artistic events exploring the possibility of creating, via DANCE paradigms of sensorial substitution, aesthetic and socially inclusive community based on the aware sharing of emotions and feeling embodied in archetypes of gestures. The gestures, transformed from an aesthetic object, into a perceptive sound object, become a social and affective catalyst.