#	Pagina
attuale pagina	/open-h2020/projects/199950/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - Terpsichore (Transforming Intangible Folkloric Performing Arts into Tangible Choreographic Digital Objects)

Teaser

Summary

This first periodic report presents the overall activities that have been carried out during the first and the second Terpsichore projectâ€™s Year. Terpsichore aims to research, analyse, develop, and implement innovative and collaborative research from heterogeneous stakeholders in multi-disciplinary sectors. The project targets at integrating the latest innovative results of photogrammetry, computer vision, semantic technologies, time evolved, 3D modeling, combined with the story telling and folklore choreography. Terpsichore involves collaboration with universities, and companies.
The overall objective of this project is to support a set of services such as virtual/augmented reality, social media, choreography database, interactive maps, presentation and learning of European Folk danced with tremendous impact on the society, culture and tourism.
The purpose of this report is to inform the stakeholders about the Terpsichore projectâ€™s achievements that have been carried out during the twenty-four months of its activity. In particular, the first periodic report includes the main achievements with respect to the key communication tools, the publications and the scientific framework that the consortium has identified and selected, collaboration opportunities pursued, awareness raising actions and the financial statements prepared and presented. In addition, in this report is presented the training framework and the transferred knowledge during the secondments and the clustering activities i.e., the relations with the International project of the Restoration of the Holy Sepulchre Church in Jerusalem, the transferring knowledge from education activities, the international activities through organization of special sessions and workshops, the Terpsichore summer schools, the secondments training activities and the built-in return mechanism for knowledge sharing and long term collaboration. Moreover, in periodic report is described the scientific result, the financial statements and the communications activities enhancing the social and scientific impact of the Terpsichore project and it is presented the future plans and the managements issues.

Work performed

The main tasks that have been achieved to reach the project objectives during the first and the second year of the project are the followings:
1) The technical specifications of the motion capturing systems have been defined.
2) The first choreographic data set was created by the Terpsichore consortium. This dataset includes data from VICON motion capturing system and Kinect-II depth sensor.
3) New algorithms for depth sensor calibration have been proposed
4) New machine learning methods including deep learning methods for dance and choreography classification and identification have been introduced
5) Novel schemes for summarising 3D joint points of dances have been developed and published in prestigious journals and conferences
6) A new tensor-based machine learning scheme that outperforms deep learning when the number of labelled data is small has been proposed
7) A new metadata framework for dance analysis and choreography has been developed and proposed
8) Many secondments carried out during the first and the second year of the project, enhancing the knowledge transfer.
9) An eye-catching website was developed including information about the Terpsichore partners and the project progress.
10) The first Terpsichore summer school was carried out in Athens with many attenders. This summer school was performed in conjunction with the famous IEEE VS-Games 2017 Conference.
11) Many publications (26) in prestigious scientific conferences have been submitted.

In addition, we have submitted all the deliverables up to the first period on the EU participant portal.

The outreach activities during this period have consisted of participation and presentations in meetings with the scientific community, with policy makers and with industry. The general public has also been reached through coverage in dissemination activities as Researcherâ€™s Night. Furthermore, quarterly leaflet is published on the project home page as a dissemination tool for the project activities.

Final results

To overcome the challenges of the innovative project, we need to perform research in the area of capturing and imaging, computer vision, 3D modeling, symbolic representation and finally virtual scene generation. In particular, the main research objectives in 3D computer vision research is to automate the 3D modeling process, while maintain high resolution accuracy. Additionally, the research is focused on complex background environments and of moving objects. To address these difficulties in the Terpsichore project, we have introduced a scalable capturing framework by incorporating state of the art devices able to acquire depth information in real-time constraints. Moreover, the current 3D modeling algorithms are not appropriate of complex human movements and complex background regions. Additionally, in case where multiple dancers interact with each other and with the environment several research challenges are emerged. To address these aspects, we need, on the one hand, computer vision tools, able to detect the foreground/background content under a highly dynamic framework, to track geometrically enriched points of interest through time and finally to estimate 3D skeletons from the 3D voxels. In addition, we need technologies able to fit the 3D skeletons into pre-defined deformable models. Furthermore, the general technical advantage of the Terpsichore project is that here the complete pipeline from capturing to digital visualization is automated due to a tremendous amount of work have been marked frame by frame in the video recordings with the aim to track the dancerâ€™s motion. Specifically, professional adult dancers and choreographers preserve and archive choreographies mainly through the use of 2D video sequences. Moreover, traditional music passes from one generation to the next either by private lesson with mentors or by hearing. In order to widen traditional music audience, inspire young musicians and preserve traditional music, during Terpsichore precise traditional music scores will be created and musical ornaments will be codified. Lyrics, that often are the result of improvisation, will be recorded to preserve local idioms and oral â€œunknownâ€ poetry. Finally, the mechanics behind traditional instruments will be explored and clarified.
We have developed new algorithms for dance summarization of data that include point joints, dance classification and posture identification algorithms including deep machine learning, deployment of a new game that can be used for dance educational purposes enhancing the social impact of the project, new calibration tools for depth sensors, a semantic metadata description framework for dances, Labanotation of the dance movements, and the creation of an innovative dance dataset.