Epoch 1️⃣5️⃣: This week in ML (+ Bioinformatics 🧬 and Astronomy 🌌)

MeshTalk, Explaining in Style, MeerCRAB, PyTorchDIA and more ...

Apr 30, 2021

Check out MeshTalk, our latest work on audio-driven face animation! Given a 3D neutral face mesh and speech as input, our approach can generate realistic lip motion and realistic upper face motion. Paper: arxiv.org/pdf/2104.08223… Video: research.fb.com/wp-content/upl…

This paper presents a generic method for generating full facial 3D animation from speech. Existing approaches to audio-driven facial animation exhibit uncanny or static upper face animation, fail to produce accurate and plausible co-articulation or rely on person-specific models that limit their scalability. To improve upon existing models, the authors proposed a generic audio-driven facial animation approach that achieves highly realistic motion synthesis results for the entire face.

Explaining in Style

Image classification models can depend on multiple different semantic attributes of the image. An explanation of the decision of the classifier needs to both discover and visualize these properties. Here the authors present StylEx, a method for doing this, by training a generative model to specifically explain multiple attributes that underlie classifier decisions. For more details please see the project page.

ML + Bioinformatics 🧬

Learning from deep learning: developing interpretable AI approaches in histopathology to predict patient prognosis and explore novel features
Abstract: Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, the authors developed a deep learning system (DLS) for predicting disease-specific survival for stage II and III colorectal cancer. When evaluated on two validation datasets, the DLS achieved a 5-year disease-specific survival AUC of 0.70 and 0.69, and added significant predictive value to a set of nine clinicopathologic features. To interpret the DLS, the authors explored the ability of different human-interpretable features to explain the variance in DLS scores. They observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores. Next, they generated human-interpretable histologic features by clustering embeddings from a deep-learning-based image-similarity model and showed that they explained the majority of the variance. Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation.
Evolving the Olfactory System with Machine Learning
The convergent evolution of the fly and mouse olfactory system led the authors to ask whether the anatomic connectivity and functional logic in vivo would evolve in artificial neural networks constructed to perform olfactory tasks. Artificial networks trained to classify odour identity recapitulate the connectivity inherent in the olfactory system. Input units are driven by a single receptor type, and units driven by the same receptor converge to form a glomerulus. Glomeruli exhibit sparse, unstructured connectivity to a larger, expansion layer. When trained to both classify odour and impart innate valence on odours, the network develops independent pathways for innate output and odour classification. Thus, artificial networks evolve even without the biological mechanisms necessary to build these systems in vivo, providing a rationale for the convergent evolution of olfactory circuits.

Astroinformatics 🌌

MeerCRAB: MeerLICHT Classification of Real and Bogus Transients using Deep Learning
Astronomers require efficient automated detection and classification pipelines when conducting large-scale surveys of the (optical) sky for variable and transient sources. Such pipelines are fundamentally important, as they permit rapid follow-up and analysis of those detections most likely to be of scientific value. In this work, the authors present a deep learning pipeline based on the convolutional neural network architecture called MeerCRAB. It is designed to filter out the so called “bogus” detections from true astrophysical sources in the transient detection pipeline of the MeerLICHT telescope.
PyTorchDIA: A flexible, GPU-accelerated numerical approach to Difference Image Analysis
The authors present a GPU-accelerated numerical approach for fast kernel and differential background solutions. The model image proposed in the Bramich (2008) difference image analysis algorithm is analogous to a very simple Convolutional Neural Network (CNN), with a single convolutional filter (i.e. the kernel) and an added scalar bias (i.e. the differential background). Here, they do not solve for the discrete pixel array in the classical, analytical linear least-squares sense. Instead, by making use of PyTorch tensors (GPU compatible multi-dimensional matrices) and associated deep learning tools, they solve for the kernel via an inherently massively parallel optimisation. By casting the Difference Image Analysis (DIA) problem as a GPU-accelerated optimisation which utilises automatic differentiation tools, their algorithm is both flexible to the choice of scalar objective function, and can perform DIA on astronomical data sets at least an order of magnitude faster than its classical analogue. More generally, they demonstrate that tools developed for machine learning can be used to address generic data analysis and modelling problems.