Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - Data4Impact (Big DATA approaches FOR improved monitoring of research and innovation performance and assessment of the societal IMPACT in the Health, Demographic Change and Wellbeing Societal Challenge)

Teaser

In the emerging context of innovation and techno-socioeconomic complex environment, the impact assessment of public-funded research and its contributions presents challenges to the funding agencies that pursue national innovation system as part of their economic growth...

Summary

In the emerging context of innovation and techno-socioeconomic complex environment, the impact assessment of public-funded research and its contributions presents challenges to the funding agencies that pursue national innovation system as part of their economic growth strategy. To be effective in the present fast changing times, we need to adopt a fact-based decision-making approach which involves putting considerable initial emphasis on the gathering of facts, figures, data and evidence.

Data4Impact is developing a set of new indicators for assessing research and innovation performance based on a hands-on and data-driven approach. It targets the Health, Demographic Change and Wellbeing societal challenge at national and EU levels, as this domain guarantees the richest availability of relative data and therefore the optimal fact checking opportunity. Through a process involving cross analysis, parallel research activities and continuous feedback loops between a methodological concept, data collection activities and data analysis, Data4Impact will a) define a framework for indicators broken down to research and innovation components and extended to all research artefacts, b) develop and apply advanced text and data mining analysis on a wide range of data (publications, patents, policy documents, funding information and reports, science media, social media, company information) and come up with a comprehensive knowledge graph that will facilitate the computation of these indicators, c) validate the proposed indicators through a series of pilots involving EU (FP7 and H2020) and three national settings (Sweden, UK, Germany) while engaging with key stakeholders via targeted outreach events and workshops throughout the process.

The foreseen outcomes of the project:
• Monitoring indicators;
• Societal impact indicators;
• Data mining and analysis algorithms;
• Data visualisation and analysis tools developed under WP6;
• Assessed performance of EU and national research and innovation systems in the new identified dimensions;
• Information on the societal impact of the analysed research activities in the Health, Demographic Change and Wellbeing societal challenge.

Work performed

The project has implemented all of its planned activities and achieved the milestones and deliverables set out in the Description of Action (DoA). WP2 and WP3 activities during this reporting period focused on landscaping and data collection (in accordance with D1.4). As a result, we have already defined the traditional monitoring impact assessment mechanisms and areas/avenues for the development of novel indicators and assessment of the societal impact of health-related research activities. Through desk research and literature review the Data4Impact team has scanned the current landscape of traditional and innovative methods, tools and practices for monitoring the performance of R&I activities and measuring their impact. Based on this, we have updated a model with a wide range of indicators/phenomena not only relating to input, throughput and output/result levels, but also the impact level. This work led to the development of the overall conceptual framework for the project (D2.1-D2.2) which has been validated and now guides our team’s work in data collection and analysis.

The WP3 team has started the process of designing and deploying the Data4Impact platform (D3.1). In WP4 and WP5 we have started to employ big data approaches for the data analysis of input, throughput, output/results and impact indicators. The key analysis methods involve topic modelling combined with deep/machine learning properties, as well as analysis of clinical guidelines data. As part of WP1, the team has started to raise awareness about the project through various outreach activities (e.g. the launch of Twitter account).

Final results

Through its activities during the first reporting period, Data4Impact has started to address the challenges as outlined in the Work Programme 2016-2017 Europe in a changing world – inclusive, innovative and reflective Societies, under the Topic “CO-CREATION-08-2016-2017: better integration of evidence on the impact of research and innovation in policy making”. In relation to these challenges, a significant impact on several sectors and layers is expected, including policymakers and research funders at national and EU levels, academic communities, researchers, but also the general public. More specifically, the work carried out under the main thematic pillars will make progress in and have an impact on the following areas:

A) A monitoring of research and innovation performance which captures the broader spectrum of ways in which research and innovation activities translate into outputs and impact

As part of WP2, Data4Impact has developed an integrated conceptual framework that builds on the state of the art in innovation research. Beyond that, the project also linked the impact dimension to the conceptual framework. We have surveyed the existing literature and have taken the findings into account when adding the impact dimension to the conceptual framework. We have also reflected on the current discussions at the European level on impact measurement and assessment when creating our concept. We are convinced that by this we are able to contribute to the structuring and clarification of the discussion on societal impact and its assessment. We have documented the concept and the literature review in an interim report and have organised an international workshop to reflect on our conceptual framework. The workshop results are feeding into the further working steps and help improving and adapting the concept. Data4Impact consortium has also worked in synergy in WP3 towards the definition of relevant input sources, identification of data analysis workflows, and integration into a common platform to enable visualisation and interrogation of project/funder quality indicators. As a result: (i) workflows have been set up and operated to produce the first indicators and (ii) the D4IP platform has been deployed. The platform (i) collects input sources to facilitate the activities of the analysis workflows (e.g. OpenAIRE projects) and (ii) aggregates the output of such analysis to support the construction of user-interfaces for search and visualisation of such indicators.

B) A reliable assessment of the societal benefits generated by public funding for research and innovation, and the impact it has on tackling major societal challenges

The Data4Impact consortium brings together information from a variety of sources and apply state-of-the-art methods to derive meaningful data and indicators for policymakers, programme funders and other stakeholders. This topic is specifically addressed through our work in all Work Packages. We have exploited data driven approaches via advanced text and data mining and network analysis, to derive evidence on the societal impact of public funding for research and innovation. This includes data on main/key technological developments in the health domain and their expected societal impact, evidence of the uptake of the research generated in clinical guidelines and practices, as well as evidence from social media and traditional science media and other sources on societal needs in the Health, Demographic Change and Wellbeing societal challenge and the contributions of public funding in meeting them.

Website & more info

More info: http://www.data4impact.eu/.