#	Pagina
attuale pagina	/open-h2020/projects/212222/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - openEO (openEO - a common, open source interface between Earth Observation data infrastructures and front-end applications)

Teaser

Summary

With a new generation of Earth Observation (EO) monitoring satellites, launched within Europe\'s Copernicus programme, the user community faces significant Big Data challenges. In fact, the capacity of the Copernicus\' Sentinel satellites to acquire data outstrips existing capacities to transmit, store, process, and analyse them. As a result, there is an urgent need to replace traditional workflows - which rely on distributing the data to thousands of users over the internet - with cloud computing approaches that bring the users and their software to the data instead. This is one of the central paradigms of the Big Data era. Many European organisations and initiatives have recognised this need at an early stage of the Copernicus programme, and are now working towards the establishment of back-ends capable of storing and processing Petabytes of Sentinel data. Since these endpoints differ regarding e.g. their data storage infrastructure, metadata standards or user interfaces, switching between back-ends is not a trivial task for the user community. The missing standardisation forms a significant entry barrier to cloud EO processing providers for the users.
Therefore, the objective of openEO is to build a common, open source interface that facilitates standardised interchange between users and back-end providers via micro-services and user-defined functions (UDF). openEO can be connected via Python, R, and JavaScript libraries, enabling to address it in the user\'s workflows. This will simplify the use of cloud-based EO processing engines, allow switching between cloud-based back office providers and comparing them, and enable reproducible, open EO science. Thereby, openEO reduces the entry barriers for the adaptation of cloud computing technologies by a broad user community and paves the way for the federation of EO data infrastructure capabilities.
The specific objectives of openEO are:
1. To establish openEO as an open source initiative by generating involvement of users and developers from both the EO and IT communities, using strong and open communication strategies
2. Define and implement an open source core API for finding, accessing, and processing large EO datasets in cloud-based data processing environments
3. Develop driver APIs to connect to back offices operated by European and worldwide industry
4. Develop open source client APIs for analysing these datasets using R, Python and JavaScript
5. Develop and publish use cases
6. Validate the openEO Interface
As openEO aims to become a widely accepted standard, its applicability shall not be limited to a few selected endpoints. Therefore, openEO will develop generic driver APIs that allow connecting to different types of back-end services, ranging from simple IaaS (Infrastructure as a Service) accounts with access to raw Copernicus (and other) data to advanced PaaS (Platform as a Service) solutions providing pre-processed Copernicus data only through platform-specific API calls. The different levels of back-end providers are represented by openEO consortium members.

Work performed

The project\'s development started with providing a Proof of Concept. This was initiated within two weeks of intense collaboration, defining and implementing micro-services and user-defined functions to create first connections from python commands to endpoint processing. For the Proof of Concept, connections were realised from the Python, R, and JavaScript clients to seven endpoints, allowing the process of micro-services and user-defined functions for three previously defined test cases.
Standards of the endpoints metadata and user interfaces were analysed to allow the further development of openEO benefitting all partners. The micro-services were expanded and standardised to create a unified openEO syntax. User\'s opinions are involved in this process. Therefore, the users are provided with several ways to contact the consortium. Also, the development of openEO happens on the freely available open source platform GitHub.
The micro-services and UDFs which are needed to implement four well defined use cases, were identified to be able to steer the project\'s further development. Meanwhile, the openEO API is constantly upgraded in various code versions, published in GitHub.

Final results

The implemented interface is already able to connect to different endpoints, using standardised commands of following topics:
- Authentication and user data management
- Query of EO data and processes available at the endpoints
- Band calculations and time series statistics
- Filtering and aggregation of specific dimensions
- UDF execution
- Zonal statistics
- Upload and use of vector data
- EO Data download
The micro-services and UDFs will be further expanded towards a version which i) is able to run four well-defined use cases at various endpoints and ii) considers user opinions to allow the future use of openEO by a broad community.
The openEO access to back-end providers represents a value adding services of EO data storage and processing services. It may empower both public and commercial users to use a standardised interface to carry out processing without having to deploy storage or processing facilities. This will enable them to reach out to a larger customer base. Users will be able to directly access back-end services and integrate them into their regular Python, R, or JavaScript workflows. The standardisation of a now highly fragmented market of non-interoperable services to make them interoperable will allow users to easily switch between endpoints. To ensure available micro-services and UDFs that reflect a whole EO data lifecycle, four well-defined use cases will be implemented, using the openEO syntax. Pilot users will evaluate these use cases to guarantee openEO\'s usability.
Since openEO is developed and distributed under a permissive open source license, both industry and society will benefit from participating together to form openEO: There are no requirements or limitations on how the developed software can be used or redistributed. Permissive licenses do not control the license terms of future products, which build on openEO. Distributing openEO explicitly under the Apache license 2.0 will simultaneously guarantee the dissemination of the open source code without any changes in the original version.