Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - FIDUCEO (Fidelity and Uncertainty in Climate data records from Earth Observations)

Teaser

Overall objective: FIDUCEO will set new standards of accuracy and rigour in the generation of Fundamental Climate Data Records (FCDRs) and Climate Data Records (CDRs) with detailed uncertainty information. This project will define and apply rigorous best practice approaches to...

Summary

Overall objective: FIDUCEO will set new standards of accuracy and rigour in the generation of Fundamental Climate Data Records (FCDRs) and Climate Data Records (CDRs) with detailed uncertainty information.

This project will define and apply rigorous best practice approaches to utilise data from the principal >20 yr-long satellite sensor series, creating 4 new FCDRs of widespread utility to science. From these, we will derive 4 datasets including CDRs of critical importance to science and society. The new FCDRs and the methods established will create world-leading European capacity for generation of rigorous CDRs across a wide range of essential climate variables (ECVs) and an exemplar template for others to follow.
The main aims are:
To create four FCDRs, for Meteosat, AVHRR, HIRS, and the microwave humidity sounder series (SSM/T2, AMSU-B, MHS), with
- physics-based harmonisation of radiance calibration
- traceable propagation of estimates of associated uncertainty components
- a standard, convenient format for wider use.

To develop and apply new harmonisation techniques across sensor-series and wavelength domains to maximise consistency of infra-red (AVHRR and HIRS) and microwave (SSM/T2, AMSU-B, MHS) FCDRs
To link thermal infra-red FCDRs (AVHRR-, HIRS- and ATSR-series) to Sentinel 3 SLSTR
To derive CDRs/new datasets from the new FCDRs for four ECVs, as valuable exemplars
- upper-tropospheric humidity (including joint evaluation of harmonised infra-red and microwave observations)
- sea and lake surface temperature
- surface albedo
- aerosol optical depth
In CDR generation, to set and apply new standards of quality, in relation to
- exploitation of new FCDR uncertainty information
- validated uncertainty budgets for every geophysical value at all spatio-temporal resolutions
- assessment of accuracy, and (crucial for CDRs) measurement stability
- convenience of access and format
To provide the wider community with re-usable and adaptable embodiments of best-practice approaches to harmonisation and uncertainty estimation for FCDR/CDRs

Work performed

The project has progressed towards its main aims and up to the end of RV2 (Month 30) has achieved the following:
FCDR’s:
• Consolidated the metrological framework. Much work has been done on this to understand how the discipline of metrology can be best used within Earth Observation and in particular for each of the 4 FCDRs. The framework has evolved from the initial concept to be focused around structured analysis of the measurement equation that defines the calibrated radiance.
In order to explain this framework, it has been important to establish a vocabulary to provide a bridge between the metrologists and the meteorologists. This vocabulary has been presented to key stakeholders and has been well received with the majority of it accepted.
• Determining the major sources of uncertainty: Using the principles of the metrological framework, each FCDR team reviewed and analysed the major sources of uncertainty in their particular sensor series. With advice from NPL on how to represent the uncertainty structure, the teams then created a series of effects tables and coded these to run on the level 1 data, creating the new FIDUCEO FCDR.
• The new FCDRs are breaking new ground in their representations of uncertainties. No standard format or definition exists for these new representations, thus a specific FIDUCEO format to accommodate this information has been created. Based on Netcdf-4, the FIDUCEO format is used in all 4 FCDRS and the format definition and specification is listed in the deliverable D3.1.
• In order to create the long term FCDRs, the separate sensor data needs to be carefully and sensitively aligned – in FIDUCEO we term this method “harmonisation” which derives the calibration parameters used in the measurement equation. The mathematics behind the harmonisation has just been finalised and will be applied to the FCDRs by the end of 2017.
• The harmonisation process also needs to determine where the actual sensors overlap. A new FIDUCEO Multi-sensor matchup system has been created to determine each of the sensor to sensor overlaps. As of October 2017, the FCDR matchups are almost complete, (1 remaining).
The next stage in the FIDUCEO FCDR process is to provide the data to beta testers. An initial small data set has been provided to one user. Further trials will be expanded over the coming months.

CDR’s:
• Creating the CDRs from the FCDRs : analysing the major sources of uncertainty. This work has started and will continue to the end of the project.
• The Multisensor matchup system will create CDR matchups and these will be used to validate the CDRs.

Outreach:
Highlights of the outreach include the EUMETSAT conference in Rome 2017 which held a FIDUCEO seminar where 5 project presentations were made and were well received. The FIDUCEO concepts will also be presented at a dedicated ESA workshop on uncertainties, October 2017.
The FIDUCEO established website has been publishing regular blogs and news stories. The effectiveness of the communication is tracked via Google Analytics.

Data Access:
The MMS software and documentation is open source and available on Github.

Final results

The FIDUCEO project will:

Integrate European scientific expertise on metrology, calibration, and retrieval science in a coherent traceable system for the production and provision of validated FCDR\'s and TCDR\'s, with the goal of contributing towards capacities in the climate change context of Copernicus.

Develop and promote methods for estimating and propagating uncertainty with the aim of providing standards for the production of CDRs for the Copernicus Climate Change Service and other European activities related to climate change.

Generate new and improved long-term FCDR\'s and associated GCOS ECV Climate Data Records applying new standards of accuracy and rigour, with the goal of providing consistent data sets containing traceable uncertainty estimates;

Provide high quality FCDR\'s and TCDR\'s, in a simple manner as an asset for the development of downstream specific climate service elements.

FIDUCEO will also contribute to several societal benefit areas of GEO/GEOSS. The provision of FCDRs will boost opportunities for deriving improved CDRs for oceanic, land surface and atmospheric parameters. With this FIDUCEO directly contributes to the GEO Climate Task CL-01 Climate Information for Adaptation that foresees the delivery of improved climate data records related to GCOS ECVs. Retrieval systems potentially applied to the FCDR outputs and also the TCDRs produced within FIDUCEO can have indirect impact on the weather, water, and agriculture societal benefit areas. FIDUCEO activities on FCDRs and TCDRs are also associated to several projects of the WMO Sustained, Coordinated Processing of Environmental Satellite Data for Climate Monitoring (SCOPE-CM) initiative where GEO is one of the stake holders.

The findings from the analysis of historic and current observations may also support the GEO Infrastructure Task IN-01 Earth Observing Systems by showcasing the best use of observations and making suggestions for future observing systems.
GEO/GEOSS is also promoting common technical standards and here FIDUCEO will provide standards on uncertainty characterisation with the Earth Observation Metrology CookBook.

Website & more info

More info: http://www.fiduceo.eu.