Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - DEDALE (Data Learning on Manifolds and Future Challenges)

Teaser

The goals of the DEDALE project were i) to introduce new models and methods to analyze and restore complex, multivariate, manifold-based signals; ii) exploit the current knowledge in optimization and operations research to build efficient numerical data processing algorithms...

Summary

The goals of the DEDALE project were i) to introduce new models and methods to analyze and restore complex, multivariate, manifold-based signals; ii) exploit the current knowledge in optimization and operations research to build efficient numerical data processing algorithms in the large-scale settings; and iii) show the reliability of the proposed data modeling and analysis technologies to tackle Scientific Big Data challenges in two different applications: one in cosmology, to map the dark matter mass map of the universe, and one in remote sensing to increase the capabilities of automatic airborne imaging analysis systems.

Work performed

The following gives a brief summary of DEDALE accomplishments presented in the context of the corresponding work packages.

WP 1: Project Management
- All requested documents (for each WP) have been delivered.
- All DEDALE journal papers are freely available on archives and the DEDALE website.
- Various meetings were organized over the course of the project.
- Several WP lead teleconferences wer organized when needed. Agendas, all presentations and minutes are available on the website.

WP 2: Low-Dimensional Models for Complex Structured Data
- We developed an adative shearlet transform and we extend it for vector-valued data.
- An inpainting algorithm for large-scale polarized data on the sphere was developed, resulting in an improved polarized E map due to a smaller T mask and cross-correlations in the TE power spectrum.
- A study on the possible extensions of dictionary learning techniques to color images was carried out, comparing learning a dictionary for RGB color images using either vectorized patches or separating the color components.
- Effort was also put in the investigation of deep learning for image enhancement problems, using either convolutional sparse coding or sparse autoencoders.
- We considered the problem of dynamic range extension, where the objective is to increase the range between the maximum and minimum values in an image.

WP 3: Signal Processing on Complex Data
- We demonstrate that the Douglas-Rachford algorithm converges to a sparse signal and give good denoising results in a multiscale representation of S1^N.
- We proposed a novel post-acquisition computational technique aiming to enhance the spectral dimensionality of imaging systems by exploiting the mathematical frameworks of Sparse Representations and Dictionary Learning.
- The key contribution of this effort is a novel Coupled Sparse Dictionary Learning model which considers coupled feature spaces, composed of low and high spectral resolution hypercubes.

WP 4: Signal Processing on Complex Data
- We developed the bGMCA algorithm for blind source separation (BSS). Experiments demonstrated that the use of intermediate block sizes dramatically enhances the separation performances compared to standard sparse BSS.
- We developed the DEDALE Distributed Learning Platform, which capable of considering both physical as well as virtual resources.
- We explored 2 use cases: (a) the super-resolution of sub-sampled hyper-spectral data (b) galaxy image deconvolution. Results show an improvement of ~60% in time response terms against the conventional computing solutions.

WP5: Applications on the Euclid Mission

- We developed a new combined deep learning and dictionary learning algorithm that reduces both the scatter and the catastrophic outlier rate in spectroscopic redshift measurements.
- A method was developed to build superresolved Euclid PSF from subsampled data. It has lead to significant improvement compared to the state of the art.
- A new deconvolution algorithm was developed for big survey images in astrophysics, which used low-rank minimization techniques, which performs well when galaxy images are not dominated by noise.
- An optimal transport was developed for PSF interpolation, results show that galaxy shapes are recovered with significantly lower errors when using our PSF model instead of PSFEx.
- We investigated the problem of automated galaxy shape and distance (redshift) estimation using non-linear deep learning models. We observe that with sufficient training time (epochs) very good prediction accuracy can be achieved.
- A mass mapping reconstruction method (GLIMPSE) has been developed which use sparse regularization.
- GLIMPSE was used to reconstruct the convergence in the field of the public Dark Energy Survey (DES) Science Verification (SV) data. The resulting mass map was shown to be considerably better than the current state-of-the-art reconstruction methods.
- We compared the existing fast-simulators and showed that

Final results

\"The DEDALE project has produced some standout results that have been disseminated via high-impact scientific journals. These include:

- A new video sequence enhancement method( see Fig. 1)
K. Fotiadou, G. Tsagkatakis, and P. Tsakalides, \"\"Spectral Resolution Enhancement of Hyperspectral Images via Sparse Representations,\"\" in Proc. 2016 IS&T International Symposium on Electronic Imaging, Computational Imaging, San Francisco, CA, February 15-19, 2016.

K. Fotiadou, G. Tsagkatakis, and P. Tsakalides. “Spectral super-resolution via coupled sparse dictionary learning.” submitted to IEEE Transactions on Computational Imaging, Special issue on Computational Imaging for Earth Sciences, 2016.

- The most accurate estimate of the cosmic microwave background from most recent available data set (WMAP-9yr and Planck-PR2), see Fig. 2
J. Bobin, F. Sureau, J-L Starck, CMB reconstruction from the WMAP and Planck PR2 data, submitted to Astronomy and Astrophysics.

- A state-of-the-art mass map produced for the Dark Energy Survey (see Fig. 3)
N. Jeffrey, F. B. Abdalla, et al., Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV, Monthly Notices of the Royal Astronomical Society, in press, 2018. DOI: 10.1093/mnras/sty1252\"

Website & more info

More info: http://dedale.cosmostat.org.