Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - LEMAN (Deep LEarning on MANifolds and graphs)

Teaser

Over the past decade, machine-learning (ML) methods have had a revolutionary impact, adding billions in business value, creating new markets, and transforming entire industrial segments. Deep learning, a particularly successful ML paradigm based on differentiable programming...

Summary

Over the past decade, machine-learning (ML) methods have had a revolutionary impact, adding billions in business value, creating new markets, and transforming entire industrial segments. Deep learning, a particularly successful ML paradigm based on differentiable programming, has been an emerging technology for decades; it took an orchestrated scientific and engineering effort to achieve an overarching technological and societal impact comparable by some experts to “modern electricity”.

Most of successful deep learning methods such as convolutional neural networks (CNNs) rely on classical signal processing models that limit their applicability to data with underlying Euclidean grid-like structure, e.g. images or acoustic signals. Yet, many applications deal with non-Euclidean (graph- or manifold-structured) data such as social networks in computational sociology, molecular graphs in chemistry, interactomes in system biology, and 3D point clouds in computer vision and graphics. Until recently, the lack of ML models capable of correctly dealing with non-Euclidean data has been a major obstacle in these fields.

The purpose of the project is to bridge the gap between geometric and ML models, in particular, developing deep learning architectures for graph- and manifold-structured data. Since graphs can model very abstract systems of relations or interactions, such methods can be applied across multiple impactful domains.

Work performed

1. Science

We have proposed some first and already popular architectures for deep learning on manifolds and graphs achieving state-of-the-art results in classically challenging problems in computer graphics and vision such as deformable dense correspondence. We also explored the use of geometric deep learning for the defense against adversarial attacks [15], recommender systems [3], and even astrophysics [10] (our paper on neutrino detection with Geometric DL in collaboration with Berkeley, NYU, and IceCube won the best paper award at ICMLA).


2. Technology

We patented our geometric deep learning technology (two granted patents [16-17], additional patents pending) and spun it off into a startup company Fabula AI funded by ERC Proof of Concept grant “GoodNews” (received by the PI in 2018) as well as additional grants from the industry
(Google, Facebook, and Amazon) and private investment. Our PhD student Federico Monti (about to graduate in 2019) took the role of CTO in the company.


3. Collaborations

The PI spend part of 2017 and 2018 on sabbatical leave at Harvard University as a Radcliffe fellow at the Institute for Advanced Study, having at the same time a visiting affiliation at MIT CSAIL. This period has brought some new research directions and allowed to forge new strong collaborations with MIT Geometry group (J. Solomon), which resulted in a series of successful publications [6-7,11] and multiple ongoing projects. We also established new collaborations with Facebook (Y. LeCun, A. Szlam), Google (A. Makadia), NYU (J. Bruna), Berkeley (Prabhat), TUM (D. Cremers, S. Guennemann), EPFL (B. Correia), Cambridge (P. Lio’), Oxford (X. Dong), Imperial College (Y.-A. de Montjoye, S. Zafeiriou), and KUL (P. Claes). We also renewed our collaboration with Stanford (L. Guibas) and Ecole Polytechnique (M. Ovsjanikov). The collaborations encompass various activities, including joint grants (e.g. with P. Claes), co-advised students (P. Lio’, S. Zafeiriou), research visits and internships (A. Makadia).


4. Dissemination

The increasing popularity of Geometric ML has seen the PI invited to speak at multiple high-level events such as the ERC Conference on Frontier Research in AI (Brussels, October 2018). The PI gave keynote talks at MICCAI Workshops on Graphs in Biomedical Image Analysis (GRAIL) and Shape in Medical Imaging (ShapeMI), Graph Signal Processing Workshop (GSP), lnternational Workshop on Differential Geometry in Computer Vision and Machine Learning (DiffCVML), lnternational Conference on 3D Vision (3DV), ECCV Workshop Geometry Meets Deep Learning (GMDL), and is scheduled to give invited talks at ICLR Workshop on Representation Learning on Graphs and Manifolds, International Conference on Medical Imaging with Deep Learning (MIDL), and International Conference on Learning, Optimization and Data (LOD).

Invited talks and seminars included top-notch institutions such as the Kavli Institute for Theoretical Physics (UCSB), Alan Turing Institute for Data Science, leading universities such as MIT, Harvard, Princeton, Yale, Oxford, and Cambrdige, and top tech companies such as Google, Facebook, and Intel.

The PI also gave invited talks at summer schools at SGP (France, 2018), MISS (Italy, 2018), and forthcoming MLSS (Moscow, 2019) and is a long-term visitor at the IPAM Long Program on Geometry and Learning at UCLA. The PI organized tutorials on geometric deep learning at NIPS 2017 (extremely popular, with nearly 3000 participants), CVPR, SIGGRAPH, and EUROGRAPHICS.

The PI has organized several conference in the domain of Geometric ML, including the series of conference Geometry Meets Deep Learning (GMDL), the IPAM Workshop on Novel Deep Learning Techniques (with Yann LeCun et al.), and held various positions as area chair (3DV 2017, ICCV 2017), and program committee member in numerous events.


5. Organized conferences

Workshop on New Deep Learning Techniques, Institute of Pure and Applied Mathematics (IPAM), UCL

Final results

Our group was among the first to bridge geometric and machine learning models, pioneering and spearheading the nascent field of Geometric Machine Learning (a term coined by our group). Our geometric deep learning algorithms have been recently used for neutrino interaction classification (a collaboration with IceCube, NYU and UC Berkeley) and fake news detection on social media (our startup Fabula AI). Our group collaborates with partners from world’s leading institutions such as Stanford, MIT, UC Berkeley, NYU, Oxford, Cambrdige, Imperial College, EPFL, Ecole Polytechnique, TUM, and Technion, as well as top industrial labs at Google, Facebook, and Intel.

Geometric deep learning is already proving a groundbreaking tool in numerous applications, including drug and material design and discovery, drug repositioning, physical sciences, and computational social sciences. Our group has produced groundbreaking results in the detection of fake news on social media from their spreading patterns using geometric deep learning. We believe that this technology, which has been spun off into a startup company Fabula AI will help to solve the deluge of misinformation plaguing the modern society.

An extremely promising direction is computational biology, where our joint work with Prof. Bruno Correia (EPFL) shows the possibility to solve notoriously hard problems in protein science, such as data-driven de novo design protein binders. This research could potentially lead to significantly and faster development of new generations of protein-based drugs (biologics) and therapies against oncological diseases.

Website & more info

More info: http://geometricdeeplearning.com.