Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - PHySIS (Sparse Signal Processing Technologies for HyperSpectral Imaging Systems)

Teaser

The next generation hyperspectral video cameras should have the ability to capture several hyperspectral data cubes per second, at almost video rates. This unprecedented wealth of information poses a major challenge and necessitates the development of highly sophisticated...

Summary

The next generation hyperspectral video cameras should have the ability to capture several hyperspectral data cubes per second, at almost video rates. This unprecedented wealth of information poses a major challenge and necessitates the development of highly sophisticated signal processing systems. The objective of this project was to develop, test, and evaluate novel signal processing technologies for real-time processing of hyperspectral data cubes. Although hyperspectral sensors capture massive amounts of high-dimensional data, relevant information usually lies in a low-dimensional space. We extended recent theoretical and algorithmic developments in the field of sparsity-enforcing recovery, compressive sensing, and matrix completion, in order to build and exploit sparse representations adapted to the hyperspectral signals of interest. All three, temporal, spatial and spectral domains of hyperspectral data were explored for sparse representations. Sparsity in the data was used not only to improve estimation performance, but also to mitigate the enormous computational burden needed to analyze hyperspectral data and leverage the development of real-time hyperspectral processing systems. The technology developed within PHySIS can be exploited to build the next generation of spaceborne remote sensing systems equipped with hyperspectral sensors to provide more efficient earth monitoring and surveillance.

Work performed

\"Within the work performed by the PHySIS consortium, the following aspects have been investigated and results have been obtained: (a) development of a testing platform for 3 Hyperspectral Imaging (HSI) cameras by IMEC; (b) investigation of the Snapshot Spectral Imaging (SSI) demosaicking problem using a low-rank Matrix Completion approach; (c) development of novel compression techniques for SSI images by designing and implementing a lightweight compression algorithm, which adapts to the specific filters arrangement on the mosaic sensor, without necessitating additional training for newly acquired HSI images; (d) investigation of HYP spectral resolution enhancement by exploiting training data for learning to map low spectral resolution cubes to higher spectral resolution ones; (e) development of HYP modelling frameworks applied on Mars Express and AVIRIS data; (f) understanding of HSI observations via joint HSI unmixing and clustering methods; (g) integration of individual H/S components into the PHySIS platform using novel processing devices like the GPU-capable NVIDIA Jetson.

The dissemination and exploitation results achieved within PHySIS include:
• 17 conference papers. The paper by G. Tsagkatakis, M. Jayapala, B. Geelen, and P. Tsakalides titled \"\"Non-negative Matrix Completion for the Enhancement of Snapshot Mosaic Multispectral Imagery\"\", won the best paper award the IS&T International Symposium on Electronic Imaging 2016, Image Sensors and Imaging Systems Conference, which took place in San Francisco on February 2016.
• 12 journal papers which were published in high impact factor journals, including papers in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Transactions on Geoscience and Remote Sensing, and Astronomy and Astrophysics among others.
• 20 talks were given at various institutes and organizations.

Important education activities in organizing workshops and meetings include:
• EUSIPCO Special Session on “Machine and Signal Learning for Big Imaging Data,” 28 August-2 September 2017
• Cosmo21: Statistical Challenges in 21st Century Cosmology, May 24-27, 2016
• Astronomical Data Analysis 2016 Summer School. May 22-24, 2016
• CosmoStat day on machine learning in Astrophysics, January 22, 2016
\"

Final results

Hyperspectral Image Acquisition: We designed and developed novel approaches for HSI acquisition. Exploiting the capabilities of the SSI technology, innovative mathematical models were developed in order to maximize the quality of the acquired imagery. To that end, three reference SSI architectures were employed, including a linescan, a 4x4 mosaic and a 5x5 mosaic. At the same time, low-rank and sparse modeling algorithmic paradigms were introduced in order to increase spatial and spectral resolution.

Sparse Representation and Compression of Hyperspectral Data: We investigated and proposed novel signal representation techniques appropriate for the compression of hyperspectral data, in conjunction with their implementation on future customized platforms. To this end, the PHySIS prototype platform was designed for jointly acquiring and compressing hyperspectral data, following the premises of compressed sensing and matrix completion. Most importantly, the developed platform, which wss built on top of an advanced hardware module (NVIDIA’s Jetson TK1 system), integrated a sophisticated lightweight encoder and a high performance decoder, in alignment with current and future HSI systems.

Sparsity-enforcing Restoration and Robust Recovery: We investigated strategies for the robust recovering of hyperspectral data, including the development of new methods which account for the presence of realistic models like impulsive noise and signal-dependent noise. To that end, various penalization schemes were explored including low-rank and spare models. The developed framework was introduced as a formal approach for the problem of Blind Source Separation which was evaluated on ESA-Planck simulated data.

Hyperspectral Image Understanding: Unmixing, Clustering, and Joint Unmixing/Clustering: Spectral unmixing is the process of separating the spectrum of a mixed measured pixel into its constituent components; the spectra of pure materials, also called endmembers, and their corresponding fractional proportions in each pixel, called abundances. We developed spectral unmixing algorithms that go beyond state-of-the-art, by exploiting not only the spectral but also the inherent spatial information of hyperspectral images. Furthermore, two novel possibilistic clustering algorithms were developed, which utilize the concept of sparsity, along with a new online clustering scheme. Last, the potential of combining the unmixing and clustering procedures was investigated with the aim to further improve clustering performance.

Integration, Demonstration, and Validation: We considered the theoretical results and prototyped algorithms of all SW processing for hyperspectral data handling to develop the engineered system integrating all the partial results. It focused on the sensing/acquisition and the novel compression/restoration/understanding techniques developed during the project, acting as the integration point of the overall research activities to obtain a complete system able to demonstrate in an end-to-end configuration the proposed functionalities and performances.

PHySIS achieved full spectral image estimates using compressed sensing reconstruction methods to process observations collected using an innovative system design. Employing novel sparse representations and matrix completion principles, we were able to accurately reconstruct spectral images with an order of magnitude more reconstructed voxels than measurements. PHySIS proposed computational hyperspectral sensor design principles will allow the next generation of hyperspectral systems to be made dramatically smaller, thus reducing their overall cost, and spark a new line of long-term technology in remote sensing and earth observation.

Website & more info

More info: http://www.physis-project.eu/.