Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - PerceptiveSentinel (Perceptive Sentinel – BIG DATA knowledge extraction and re-creation platform)

Teaser

Free and open access to high temporal and high spatial resolution Copernicus’ earth observation (hereafter: EO) data is becoming a major game-changer in EO sector, delivering new OPPORTUNITIES as well as new CHALLENGES at the same time. How to ingest enormous amounts of data...

Summary

Free and open access to high temporal and high spatial resolution Copernicus’ earth observation (hereafter: EO) data is becoming a major game-changer in EO sector, delivering new OPPORTUNITIES as well as new CHALLENGES at the same time. How to ingest enormous amounts of data (BIG-DATA), how to unlock the hidden value of the data and how to deliver new value to end user community, remain unanswered questions up to date. At this place, it has to be emphasised that consortium members already embarked this mission before this Call was published (alone or in partnership) and are considering this project as a logical continuation of their existent efforts, alignment with this Call just being a welcome and advantageous circumstance. The fact that this project is not starting from the scratch is setting project’s ambition quite high. Project deliveries will be built upon two existing BIG DATA solutions (1) award-winning BIG DATA solution: Sentinel Hub (LINK to 2016 Copernicus Masters Award; winner in BIG-DATA category + Overall winner); world-first engine for archiving, processing and distribution of SENTINEL-2 data and (2) EO-Toolset, world’s first EO intermediate platform. Our goal is for project “Perceptive Sentinel – BIG DATA knowledge extraction and re-creation platform” (hereafter: PerceptiveSentinel project) to challenge the current EO exploitation practices by delivering completely new, revolutionary EO ecosystem service, providing the answers to key challenges:


The project objective is to build “Perceptive Sentinel – BIG DATA knowledge extraction and re-creation platform” (hereafter: eo - learn) to challenge the current EO exploitation practices by delivering completely new, revolutionary EO ecosystem. The availability of open Earth Observation (EO) data through the Copernicus and Landsat programs represents an unprecedented resource for many EO applications, ranging from land use and land cover (LULC) monitoring, crop monitoring and yield prediction, to disaster control, emergency services and humanitarian relief. Given the large amount of high spatial resolution data at high revisit frequency, frameworks able to automatically extract complex patterns in such spatio-temporal data are required. Eo-learn as PerceptiveSentinel aims at providing a set of tools to make prototyping of complex EO workflows as easy, fast, and accessible as possible.

Work performed

During the first reporting period the initial focus of the project was on providing data analysis, data collection and data verification. We have uploaded all the EO-data as well as non-EO data to the project FTP server, so that they can be used et masse. The data are accessible using eo-learn platform as well. For verification of EO and non-EO data purposes, data from field trials studying N application rates in winter wheat were obtained. The Algorithm Theoretic Baseline Document was developed and contains an overview of to be image processing algorithms that are going to be implemented and summarizes necessary vocabulary on data, processing levels, algorithms and remote sensing as well as pre-processing methods. Selected features are mainly spatial or local parameters such as texture values, shape and geometric descriptors. Spatial and radiometric solution were selected from the state of the art by its characteristics of performance and low computing time, in order to be extracted systematically by the eo - learn platform for deriving higher level information. We have prepared working prototypes and pushed the feature extractors into the PreceptiveSentinel eo-learn platform. We have produced and refined the top-level architectural design of the eo-learn platform, i.e. the top-level structure and software components.

In the first period of the project, eo-learn has grown into a remarkable piece of open-source software, ready to be put to use by anyone who is curious about EO data, to actually using them for data science and machine learning. We did a blog series on land use and land cover classification, using eo-learn , which have accumulated more than 25 thousand views so far.
On GitHub , open-source repository, where eo-learn is available for download, the project has over 230 stars and 35 developers are subscribed to notifications about changes of eo-learn. More than 60 developers have created their own branch of eo-learn, which people do when they want to do local modifications. These numbers do not include people, who use eo-learn as a library installed like any other Python package. The eo-learn repository has more than 2.500 views in a bi-weekly period coming from more than 400 unique users.

The Sentinel Hub forum thread dedicated to our open-source libraries such as eo-learn has over 50 threads, which were viewed more than 10 thousand times. There are at least 10 Sentinel Hub users, who are paying Enterprise level subscription, for whom we know they are using it due to eo-learn. One of these is one of the largest mapping companies in the world, another one is one of the largest European EO companies. Between 10 and 30 million requests to Sentinel Hub service come from people using eo-learn.

Final results

We believe that the best measure for the impact is how much the tools, resulting from the project, are being used. This measure has exceeded our expectations significantly, especially due to the fact that project is barely in half-time. Since the project results are available on GitHub for anyone to use, without registration, we do not have exact information on all users. However, the information we have, are remarkable:
-Active engagement of the users on GitHub (275+ stars, 39 subscriptions for notifications, 75+ branches).
-More than 30.000 reads of the eo-learn blog posts series.
- Between 10 and 30 million requests processed every month by Sentinel Hub coming from eo-learn scripts, both from research as well as commercial users (including large corporations). Due to the open nature of the platform, we do not have full visibility on what users are exploring with eo-learn. We have however seen many examples on land cover classification and deforestation.
-The eo-learn initiative being recognised as “community contribution” by Group on Earth Observation (GEO), part of United Nations. In order to support exploitation of EO data in developing countries, Sinergise partnered with GEO providing significant volume of processing services .
-Best poster award on Big data from Space conference.
-Several workshops accepted focused specifically to train people on PerceptiveSentinel platform, eo-learn.

Website & more info

More info: http://www.perceptivesentinel.eu/.