Deep Learning
Deep Learning Prediction of Effectors
A tool that contains trained deep learning models for predicting effector proteins. deepredeff has been trained to identify effector proteins using a set of known experimentally validated effectors from either bacteria, fungi, or oomycetes. Documentation is available via several vignettes, and the paper by Kristianingsih and MacLean (2020) .
AI Can Compute Protein Structures in 10 Minutes
Scientists have waited months for access to high-accuracy protein structure prediction since DeepMind presented remarkable progress in this area at the 2020 Critical Assessment of Structure Prediction, or CASP14, conference. The wait is now over. Researchers at the Institute for Protein Design at the University of Washington School of Medicine in Seattle have largely recreated the performance achieved by DeepMind on this important task. These results will be published by the journal Science. Unlike DeepMind, the UW Medicine team has already made their method, dubbed RoseTTAFold, freely available.
Neural Network for Predicting the Energy Performance of a Building
I spent years, during my master's studies in engineering, trying to model energy systems. In most cases, only the simplest problems can be modelled directly (analytical resolution of the governing differential equations of the system studied), for particularly simple and convenient geometries and boundary conditions. More complex problems are tackled using various mathematical/numerical or procedural techniques to simplify their nature so that reasonably accurate, albeit approximate, calculation models can be developed. One of the most interesting aspects of my recent experience with the integration of Deep Learning and engineering in the broadest sense (structural analysis, fluid dynamics, energy systems…) is the possibility of approaching the problems studied in completely different ways. The abundance and complexity of data is no longer a problem, but an advantage, allowing more accurate and sophisticated forecasting models to be developed.
OpenAI Codex shows the limits of large language models
This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence. In a new paper, researchers at OpenAI have revealed details about Codex, a deep learning model that generates software source code. Codex powers Copilot, an "AI pair programmer" tool developed jointly by OpenAI and GitHub. Copilot is currently available in beta test mode to a limited number of users. The paper is a fascinating read that explains the process through which the scientists at OpenAI managed to repurpose their flagship language model GPT-3 to create Codex.
Top Machine Learning Funding and Investments in Q2 2021
From voice assistants to self-driving cars, artificial intelligence and machine learning are overtaking every aspect of the industrial sector. Machine learning algorithms are used to automate laborious tasks in businesses to discover patterns in existing data without being explicitly programmed. The field is continuously evolving and high-value predictions are being used to make better decisions in real-time without human interventions. Under recent circumstances, investments in machine learning companies have drastically increased. Analytics Insight presents the top machine learning funding and investments in Q2 2021.
Constrained Feedforward Neural Network Training via Reachability Analysis
Chung, Long Kiu, Dai, Adam, Knowles, Derek, Kousik, Shreyas, Gao, Grace X.
Neural networks have recently become popular for a wide variety of uses, but have seen limited application in safety-critical domains such as robotics near and around humans. This is because it remains an open challenge to train a neural network to obey safety constraints. Most existing safety-related methods only seek to verify that already-trained networks obey constraints, requiring alternating training and verification. Instead, this work proposes a constrained method to simultaneously train and verify a feedforward neural network with rectified linear unit (ReLU) nonlinearities. Constraints are enforced by computing the network's output-space reachable set and ensuring that it does not intersect with unsafe sets; training is achieved by formulating a novel collision-check loss function between the reachable set and unsafe portions of the output space. The reachable and unsafe sets are represented by constrained zonotopes, a convex polytope representation that enables differentiable collision checking. The proposed method is demonstrated successfully on a network with one nonlinearity layer and approximately 50 parameters.
Data-informed Deep Optimization
Zhang, Lulu, Xu, Zhi-Qin John, Zhang, Yaoyu
Complex design problems are common in the scientific and industrial fields. In practice, objective functions or constraints of these problems often do not have explicit formulas, and can be estimated only at a set of sampling points through experiments or simulations. Such optimization problems are especially challenging when design parameters are high-dimensional due to the curse of dimensionality. In this work, we propose a data-informed deep optimization (DiDo) approach as follows: first, we use a deep neural network (DNN) classifier to learn the feasible region; second, we sample feasible points based on the DNN classifier for fitting of the objective function; finally, we find optimal points of the DNN-surrogate optimization problem by gradient descent. To demonstrate the effectiveness of our DiDo approach, we consider a practical design case in industry, in which our approach yields good solutions using limited size of training data. We further use a 100-dimension toy example to show the effectiveness of our model for higher dimensional problems. Our results indicate that the DiDo approach empowered by DNN is flexible and promising for solving general high-dimensional design problems in practice.
Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
Moseley, Ben, Markham, Andrew, Nissen-Meyer, Tarje
Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work we propose a new, scalable approach for solving large problems relating to differential equations called Finite Basis PINNs (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.
Deep Learning Beam Optimization in Millimeter-Wave Communication Systems
Ismayilov, Rafail, Cavalcante, Renato L. G., Stańczak, Sławomir
We propose a method that combines fixed point algorithms with a neural network to optimize jointly discrete and continuous variables in millimeter-wave communication systems, so that the users' rates are allocated fairly in a well-defined sense. In more detail, the discrete variables include user-access point assignments and the beam configurations, while the continuous variables refer to the power allocation. The beam configuration is predicted from user-related information using a neural network. Given the predicted beam configuration, a fixed point algorithm allocates power and assigns users to access points so that the users achieve the maximum fraction of their interference-free rates. The proposed method predicts the beam configuration in a "one-shot" manner, which significantly reduces the complexity of the beam search procedure. Moreover, even if the predicted beam configurations are not optimal, the fixed point algorithm still provides the optimal power allocation and user-access point assignments for the given beam configuration.
Explainable AI Enabled Inspection of Business Process Prediction Models
Ouyang, Chun, Sindhgatta, Renuka, Moreira, Catarina
Modern data analytics underpinned by machine learning techniques has become a key enabler to the automation of data-led decision making. As an important branch of state-of-the-art data analytics, business process predictions are also faced with a challenge in regard to the lack of explanation to the reasoning and decision by the underlying `black-box' prediction models. With the development of interpretable machine learning techniques, explanations can be generated for a black-box model, making it possible for (human) users to access the reasoning behind machine learned predictions. In this paper, we aim to present an approach that allows us to use model explanations to investigate certain reasoning applied by machine learned predictions and detect potential issues with the underlying methods thus enhancing trust in business process prediction models. A novel contribution of our approach is the proposal of model inspection that leverages both the explanations generated by interpretable machine learning mechanisms and the contextual or domain knowledge extracted from event logs that record historical process execution. Findings drawn from this work are expected to serve as a key input to developing model reliability metrics and evaluation in the context of business process predictions.