Epoch 2️⃣2️⃣: This week in ML (+ Bioinformatics 🧬 and Astronomy 🌌)
Principles of Deep Learning Theory, Graph Neural Diffusion, SnakeLines, and more ..
The Principles of Deep Learning Theory will be published by Cambridge University Press in early 2022 and the manuscript is now publicly available. This is only the first step toward the much larger project of reimagining a science of AI, one that’s both derived from first principles and at the same time focused on describing how realistic models actually work. If successful, such a general theory of deep learning could potentially enable vastly more powerful AI models and perhaps even guide us toward a framework for studying universal aspects of intelligence.
The authors present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE. In their model, the layer structure and topology correspond to the discretisation choices of temporal and spatial operators. Their approach allows a principled development of a broad new class of GNNs that are able to address the common plights of graph learning models such as depth, over-smoothing, and bottlenecks. Key to the success of our models are stability with respect to perturbations in the data and this is addressed for both implicit and explicit discretisation schemes. They develop linear and nonlinear versions of GRAND, which achieve competitive results on many standard graph benchmarks. Link 🔗 to the code.
ML in Bioinformatics 🧬
The visual system of mammals is comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. In particular, the clearest example is the specialization of the ventral (“what”) and dorsal (“where”) pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. Here, the authors ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. They show that when we train a deep neural network architecture with two parallel pathways using a self-supervised predictive loss function, they can outperform other models in fitting mouse visual cortex. Moreover, they can model both the dorsal and ventral pathways.
SnakeLines: integrated set of computational pipelines for sequencing reads
With decreasing price of massive parallel sequencing technologies, more and more laboratories are utilizing resulting sequences of DNA fragments for genomic analysis. A substantial obstacle for interpretation is transforming of sequenced data into results interpretable by clinicians and researchers without computational background. Laboratories are generally using computational pipelines consisting of several bioinformatic tools. The authors propose several computational pipelines for processing of paired-end Illumina reads; including mapping, assembly, variant calling, viral identification, RNA-seq and metagenomics analysis. All provided pipelines are embedded into virtual environments that ensures isolation of required resources from host operating system, rapid deployment and reproducibility of results across different platforms. Link to the code.
Astroinformatics 🌌
Unsupervised Resource Allocation with Graph Neural Networks
The authors present an approach for maximizing a global utility function by learning how to allocate resources in an unsupervised way. They expect interactions between allocation targets to be important and therefore propose to learn the reward structure for near-optimal allocation policies with a GNN. By relaxing the resource constraint, we can employ gradient-based optimization in contrast to more standard evolutionary algorithms. Their algorithm is motivated by a problem in modern astronomy, where one needs to select-based on limited initial information-among 109 galaxies those whose detailed measurement will lead to optimal inference of the composition of the universe. They anticipate that their technique will also find applications in a range of resource allocation problems. Code to be available soon.
Reducing Effects of Swath Gaps on Unsupervised Machine Learning Models for NASA MODIS Instruments
Due to the nature of their pathways, NASA Terra and NASA Aqua satellites capture imagery containing swath gaps, which are areas of no data. Swath gaps can overlap the region of interest (ROI) completely, often rendering the entire imagery unusable by Machine Learning (ML) models. The authors propose an augmentation technique that considerably removes the existence of swath gaps in order to allow CNNs to focus on the ROI, and thus successfully use data with swath gaps for training. Additionally, they perform qualitative analysis using activation maps that visualizes the effectiveness of our trained network in not paying attention to the swath gaps. Link 🔗 to the code.
Explainable, Interpretable, Bias and Ethics in AI
Interesting Events and News 📰
Hubble's Main Computer Is Offline, And NASA Is Desperately Attempting to Fix It
Indian AI startups Qure.ai & Wysa win NHS AI Lab's AI in Health & Care Award
Tesla unveils its new supercomputer (5th most powerful in the world) to train self-driving AI
Articles and Resources 📃 I liked
[Misc] How These Three Types of Writing Can Improve Self-Awareness And Mental Health
🎥 [CVPR 2021 Tutorial] Leave Those Nets Alone: Advances in Self-Supervised Learning
Recommended Podcasts 🎧
Gradient Dissent | Bringing genetic insights to everyone with Invitae's Head of AI, Matthew Davis
TWIML AI Podcast | AI and Society: Past, Present and Future with Eric Horvitz - #493