Deep Learning
DeepMind's AI solved a 50-year-old protein-related challenge - TechStory
DeepMind has recently solved a 50-year-old protein-related challenge that has notched up the level of biology for scientists. AI can help us understand the world and DeepMind has proof. Today, DeepMind in association with CASP or Critical Assessment of protein Structure Prediction competition announced an Artificial Intelligence that should have a huge impact on biology and the human efforts to understand life. We are talking about DeepMind's AlphaFold, the latest iteration, a deep-learning system that can accurately predict the structure of proteins within the size of an atom. This is a major development in biology as this has been a challenge since the 1970s, almost 50 years old challenge has today been solved by an Artificial Intelligence system.
From Elon's mind to Bill Gate's wallet: How GPT-3 ended up on Azure
Microsoft recently announced it will soon offer'invitation only' access to GPT-3 via Azure. This is a weird bit of news. We all saw it coming the moment Microsoft tossed Open AI a cool $1B for "pre AGI technologies" (AGI, artificial general intelligence, doesn't exist yet, so literally everything is a pre AGI tech lol). But it's unclear exactly what this means for OpenAI going forward. Back in 2019, when the two companies inked the partnership, it seemed like OpenAI was going to beef up Azure's backbone. What we're seeing today is more likeโฆ a turn-key business opportunity.
TinyML Enabling Low-Power Inferencing, Analytics at the Edge - AI Trends
Edge computing is booming, with estimates ranging up to $61 billion in value in 2028. While definitions vary, edge computing is about taking compute power out of the data center and bringing it as close as possible to the device where analytics can run. The devices can be standalone IoT sensors, drones, or autonomous vehicles. Increasingly, data generated at the edge are used to feed applications powered by machine learning models," stated George Anadiotis, analyst, engineer and founder of Linked Data Orchestration of Berlin, Germany, working on the intersection of technology, media and data, writing in a recent account in ZDnet. However, "There's just one problem: machine learning models were never designed to be deployed at the edge.
FEAFA+: An Extended Well-Annotated Dataset for Facial Expression Analysis and 3D Facial Animation
Gan, Wei, Xue, Jian, Lu, Ke, Yan, Yanfu, Gao, Pengcheng, Lyu, Jiayi
Nearly all existing Facial Action Coding System-based datasets that include facial action unit (AU) intensity information annotate the intensity values hierarchically using A--E levels. However, facial expressions change continuously and shift smoothly from one state to another. Therefore, it is more effective to regress the intensity value of local facial AUs to represent whole facial expression changes, particularly in the fields of expression transfer and facial animation. We introduce an extension of FEAFA in combination with the relabeled DISFA database, which is available at https://www.iiplab.net/feafa+/ now. Extended FEAFA (FEAFA+) includes 150 video sequences from FEAFA and DISFA, with a total of 230,184 frames being manually annotated on floating-point intensity value of 24 redefined AUs using the Expression Quantitative Tool. We also list crude numerical results for posed and spontaneous subsets and provide a baseline comparison for the AU intensity regression task.
Relative stability toward diffeomorphisms indicates performance in deep nets
Petrini, Leonardo, Favero, Alessandro, Geiger, Mario, Wyart, Matthieu
Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this question by defining a maximum-entropy distribution on diffeomorphisms, that allows to study typical diffeomorphisms of a given norm. We confirm that stability toward diffeomorphisms does not strongly correlate to performance on benchmark data sets of images. By contrast, we find that the stability toward diffeomorphisms relative to that of generic transformations $R_f$ correlates remarkably with the test error $\epsilon_t$. It is of order unity at initialization but decreases by several decades during training for state-of-the-art architectures. For CIFAR10 and 15 known architectures, we find $\epsilon_t\approx 0.2\sqrt{R_f}$, suggesting that obtaining a small $R_f$ is important to achieve good performance. We study how $R_f$ depends on the size of the training set and compare it to a simple model of invariant learning.
A Cyber Threat Intelligence Sharing Scheme based on Federated Learning for Network Intrusion Detection
Sarhan, Mohanad, Layeghy, Siamak, Moustafa, Nour, Portmann, Marius
The uses of Machine Learning (ML) in detection of network attacks have been effective when designed and evaluated in a single organisation. However, it has been very challenging to design an ML-based detection system by utilising heterogeneous network data samples originating from several sources. This is mainly due to privacy concerns and the lack of a universal format of datasets. In this paper, we propose a collaborative federated learning scheme to address these issues. The proposed framework allows multiple organisations to join forces in the design, training, and evaluation of a robust ML-based network intrusion detection system. The threat intelligence scheme utilises two critical aspects for its application; the availability of network data traffic in a common format to allow for the extraction of meaningful patterns across data sources. Secondly, the adoption of a federated learning mechanism to avoid the necessity of sharing sensitive users' information between organisations. As a result, each organisation benefits from other organisations cyber threat intelligence while maintaining the privacy of its data internally. The model is trained locally and only the updated weights are shared with the remaining participants in the federated averaging process. The framework has been designed and evaluated in this paper by using two key datasets in a NetFlow format known as NF-UNSW-NB15-v2 and NF-BoT-IoT-v2. Two other common scenarios are considered in the evaluation process; a centralised training method where the local data samples are shared with other organisations and a localised training method where no threat intelligence is shared. The results demonstrate the efficiency and effectiveness of the proposed framework by designing a universal ML model effectively classifying benign and intrusive traffic originating from multiple organisations without the need for local data exchange.
Tea Chrysanthemum Detection under Unstructured Environments Using the TC-YOLO Model
Qi, Chao, Gao, Junfeng, Pearson, Simon, Harman, Helen, Chen, Kunjie, Shu, Lei
These authors contributed equally to this work and should be considered co-first authors. Abstract: Tea chrysanthemum detection at its flowering stage is one of the key components for selective chrysanthemum harvesting robot development. However, it is a challenge to detect flowering chrysanthemums under unstructured field environments given the variations on illumination, occlusion and object scale. In this context, we propose a highly fused and lightweight deep learning architecture based on YOLO for tea chrysanthemum detection (TC-YOLO). First, in the backbone component and neck component, the method uses the Cross-Stage Partially Dense Network (CSPDenseNet) as the main network, and embeds custom feature fusion modules to guide the gradient flow. In the final head component, the method combines the recursive feature pyramid (RFP) multiscale fusion reflow structure and the Atrous Spatial Pyramid Pool (ASPP) module with cavity convolution to achieve the detection task. The resulting model was tested on 300 field images, showing that under the NVIDIA Tesla P100 GPU environment, if the inference speed is 47.23 FPS for each image (416 416), TC-YOLO can achieve the average precision (AP) of 92.49% on our own tea chrysanthemum dataset. In addition, this method (13.6M) can be deployed on a single mobile GPU, and it could be further developed as a perception system for a selective chrysanthemum harvesting robot in the future. Keywords: Tea chrysanthemum; Flowering stage detection; Deep convolutional neural network; Agricultural robotics 1. Introduction Current studies show that tea chrysanthemums have significant commercial value (Liu et al., 2020; Liu et al., 2019). Not only that, but tea chrysanthemums can offer a range of health benefits (Hou et al., 2017; Yue et al., 2018). For example, they can significantly inhibit the activity of carcinogens and have distinct anti-aging, cholagogic and antihypertensive effects (Zheng et al., 2021).
Identifying nonlinear dynamical systems from multi-modal time series data
Bommer, Philine Lou, Kramer, Daniel, Tombolini, Carlo, Koppe, Georgia, Durstewitz, Daniel
Empirically observed time series in physics, biology, or medicine, are commonly generated by some underlying dynamical system (DS) which is the target of scientific interest. There is an increasing interest to harvest machine learning methods to reconstruct this latent DS in a completely data-driven, unsupervised way. In many areas of science it is common to sample time series observations from many data modalities simultaneously, e.g. electrophysiological and behavioral time series in a typical neuroscience experiment. However, current machine learning tools for reconstructing DSs usually focus on just one data modality. Here we propose a general framework for multi-modal data integration for the purpose of nonlinear DS identification and cross-modal prediction. This framework is based on dynamically interpretable recurrent neural networks as general approximators of nonlinear DSs, coupled to sets of modality-specific decoder models from the class of generalized linear models. Both an expectation-maximization and a variational inference algorithm for model training are advanced and compared. We show on nonlinear DS benchmarks that our algorithms can efficiently compensate for too noisy or missing information in one data channel by exploiting other channels, and demonstrate on experimental neuroscience data how the algorithm learns to link different data domains to the underlying dynamics