Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - SUITCEYES (Smart, User-friendly, Interactive, Tactual, Cognition-Enhancer that Yields Extended Sensosphere - Appropriating sensor technologies, machine learning, gamification and smart haptic interfaces)

Teaser

The overall objective of SUITCEYES is to improve the level of independence and participation of persons with deafblindness and to enhance their communication, perception of the environment, knowledge acquisition, and conduct of daily routines.SUITCEYES proposes a new...

Summary

The overall objective of SUITCEYES is to improve the level of independence and participation of persons with deafblindness and to enhance their communication, perception of the environment, knowledge acquisition, and conduct of daily routines.

SUITCEYES proposes a new, intelligent, flexible and expandable mode of haptic communication via soft interfaces. It brings user needs and policy studies together with disability studies, haptics, psychophysics, smart textiles, sensor technology, semantics, face and object recognition, image processing, and gamification in order to address the following three challenges: (a) perception of the environment; (b) communication and exchange of semantic content; (c) joyful learning and life experiences.

The right to education, good standards of living, and other fundamental human rights for all people, including people with disabilities, have been proclaimed both by the United Nations and European Union. However, assistive technologies to facilitate communication for people with deafblindness remain limited. Difficulty in communication affects all aspects of life. The haptic communication interface, which will enable improved communication, will be a positive step in supporting people with deafblindness to achieve more of their aspirations. At the same time, the technology developed in the project will facilitate remote communication between the users and their families, support systems and educators. Availability of improved communication solutions (i.e. from distance rather than through, for example, tactile signs from person to person) will also enhance the user\'s ability to be a more active participant in society, and hence society at large will benefit from the contributions of the intended users of the technologies developed in the project.

Work performed

So far within the project
(A) A large user-study has been conducted and analysed. The overall aim of the study was to develop an understanding of whether and how new haptic technologies might be useful in everyday life and learning situations, as well as the types of opportunities and barriers that affect their availability and use by persons with deafblindness. Based on these, a set of user requirements have been identified and a number of use-case scenarios formulated.
(B) Sophisticated algorithms have been developed for real-time object, face and scene detection and recognition.
(C) Work has started on semantic representation and reasoning and experimenting with haptic solutions that can mimic social haptic communication which is conducted by the use of hand gestures on the body.
(D) Sensors have been reviewed and most relevant options have been identified.
(E) Prototypes have been developed that detect obstacles and obstacle-free spaces and Smart textile prototypes that accommodate related sensors and actuators in a comfortable, discrete, and well-presented fashion.
(F) Gamified scenarios have been identified and experimented with.
(G) All these efforts have been tested in psychological experiments for relevance and effectiveness.

Final results

To date, very little research has involved people with deafblindness and our user study is one of the largest and most comprehensive of its kind, involving 81 interviews with 79 participants in 5 EU countries. From this experience, and drawing on previous work, we have developed guidance on good practice in interviewing and involving people with deafblindness in research. From the data generated, we are developing a detailed picture of the priorities, needs, and barriers regarding access to new technologies and their use by this group of people. Working closely with other work packages, we have identified feasible and important functions for the SUITCEYES technology to fulfil. At a time of rapid technological development, the project is well placed to inform current important work on the effective and ethical use and regulation of new technologies in society.

A key aim of WP3 is to develop algorithms for extracting meaningful information from visual input (i.e. camera feed) and, more specifically, to detect and recognize objects and faces in the user’s environment in a computationally efficient fashion. A further aim is to develop a semantic model (also called an “ontology”) for (a) semantically integrating and representing information received from the system’s analysis components (WP3-4) in a uniform manner, and, (b) performing rule-based reasoning that will derive implicit knowledge from explicitly asserted facts.
During the reporting period, we made substantial progress on both fronts. Regarding visual analysis, we have been developing algorithms for real-time detection and recognition of objects, faces, scenes, and activities with high accuracy using sophisticated dimensionality reduction and machine learning approaches. With respect to semantics, we have developed a full-fledged ontology-based model that can very flexibly represent various aspects. Moreover, we have successfully integrated the two components, while we are also working together with the other partners on integrating the rest of the components (WP4) with the ontology.
With regards to WP7 and gamification:
- We now have a better understanding of the concept of play and playfulness of persons with deafblindness
- We derived concepts of gamified scenarios tailored to individuals with deafblindness, designed to enrich experiences, gamify learning of HIPI and enhance social experiences
- We implemented a first gamified concept for individuals with deafblindness (Easter-Egg-Hunt-Scenario, using a Wizard-of-Oz-approach to compensate for missing sensoric input)

WP8 expects the following results until the end of the project and potential impacts:
- reaching broad target audiences as defined in the project (research and academic, industry and interest-groups, general public and policymakers) to inform about the project, and engaging users and stakeholders in developing intended solutions.
- ensuring that IP rights are not violated while sharing SUITCEYES findings with other research groups.
- implementation of affordable wearable products and technologies with sensors, actuators, and haptic devices.
- allowing users to become more independent and more productive, resulting in reduced costs for care (business case), while at the same time enriching life experiences.
- focusing on expansion in the most important markets, trying to provide a platform with a large technological and market potential in relation to competitive solutions.
- participation in external fairs and exhibitions devoted to the subject of deafblindness to promote the platform and hold meetings with potential contractors and industry organizations.

By the end of the project, we will be able to present a platform for haptic communication specifically designed for use by people with deafblindness that can be further developed by other interested parties. The technologies developed within the project are expected to act as a complement to existing communication modes, affording people

Website & more info

More info: http://www.suitceyes.eu.