Oceania
A Convolutional Neural Network Approach to Supernova Time-Series Classification
Qu, Helen, Sako, Masao, Moller, Anais, Doux, Cyrille
One of the brightest objects in the universe, supernovae (SNe) are powerful explosions marking the end of a star's lifetime. Supernova (SN) type is defined by spectroscopic emission lines, but obtaining spectroscopy is often logistically unfeasible. Thus, the ability to identify SNe by type using time-series image data alone is crucial, especially in light of the increasing breadth and depth of upcoming telescopes. We present a convolutional neural network method for fast supernova time-series classification, with observed brightness data smoothed in both the wavelength and time directions with Gaussian process regression. We apply this method to full duration and truncated SN time-series, to simulate retrospective as well as real-time classification performance. Retrospective classification is used to differentiate cosmologically useful Type Ia SNe from other SN types, and this method achieves >99% accuracy on this task. We are also able to differentiate between 6 SN types with 60% accuracy given only two nights of data and 98% accuracy retrospectively.
Bayesian Generational Population-Based Training
Wan, Xingchen, Lu, Cong, Parker-Holder, Jack, Ball, Philip J., Nguyen, Vu, Ru, Binxin, Osborne, Michael A.
Reinforcement learning (RL) offers the potential for training generally capable agents that can interact autonomously in the real world. However, one key limitation is the brittleness of RL algorithms to core hyperparameters and network architecture choice. Furthermore, non-stationarities such as evolving training data and increased agent complexity mean that different hyperparameters and architectures may be optimal at different points of training. This motivates AutoRL, a class of methods seeking to automate these design choices. One prominent class of AutoRL methods is Population-Based Training (PBT), which have led to impressive performance in several large scale settings. In this paper, we introduce two new innovations in PBT-style methods. First, we employ trust-region based Bayesian Optimization, enabling full coverage of the high-dimensional mixed hyperparameter search space. Second, we show that using a generational approach, we can also learn both architectures and hyperparameters jointly on-the-fly in a single training run. Leveraging the new highly parallelizable Brax physics engine, we show that these innovations lead to large performance gains, significantly outperforming the tuned baseline while learning entire configurations on the fly. Code is available at https://github.com/xingchenwan/bgpbt.
FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations
Ribeiro, Leonardo F. R., Liu, Mengwen, Gurevych, Iryna, Dreyer, Markus, Bansal, Mohit
Despite recent improvements in abstractive summarization, most current approaches generate summaries that are not factually consistent with the source document, severely restricting their trust and usage in real-world applications. Recent works have shown promising improvements in factuality error identification using text or dependency arc entailments; however, they do not consider the entire semantic graph simultaneously. To this end, we propose FactGraph, a method that decomposes the document and the summary into structured meaning representations (MR), which are more suitable for factuality evaluation. MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form, and reducing data sparsity. FactGraph encodes such graphs using a graph encoder augmented with structure-aware adapters to capture interactions among the concepts based on the graph connectivity, along with text representations using an adapter-based text encoder. Experiments on different benchmarks for evaluating factuality show that FactGraph outperforms previous approaches by up to 15%. Furthermore, FactGraph improves performance on identifying content verifiability errors and better captures subsentence-level factual inconsistencies.
Over-the-Air Federated Edge Learning with Hierarchical Clustering
Aygün, Ozan, Kazemi, Mohammad, Gündüz, Deniz, Duman, Tolga M.
We examine federated learning (FL) with over-the-air (OTA) aggregation, where mobile users (MUs) aim to reach a consensus on a global model with the help of a parameter server (PS) that aggregates the local gradients. In OTA FL, MUs train their models using local data at every training round and transmit their gradients simultaneously using the same frequency band in an uncoded fashion. Based on the received signal of the superposed gradients, the PS performs a global model update. While the OTA FL has a significantly decreased communication cost, it is susceptible to adverse channel effects and noise. Employing multiple antennas at the receiver side can reduce these effects, yet the path-loss is still a limiting factor for users located far away from the PS. To ameliorate this issue, in this paper, we propose a wireless-based hierarchical FL scheme that uses intermediate servers (ISs) to form clusters at the areas where the MUs are more densely located. Our scheme utilizes OTA cluster aggregations for the communication of the MUs with their corresponding IS, and OTA global aggregations from the ISs to the PS. We present a convergence analysis for the proposed algorithm, and show through numerical evaluations of the derived analytical expressions and experimental results that utilizing ISs results in a faster convergence and a better performance than the OTA FL alone while using less transmit power. We also validate the results on the performance using different number of cluster iterations with different datasets and data distributions. We conclude that the best choice of cluster aggregations depends on the data distribution among the MUs and the clusters.
$\ell_\infty$-Robustness and Beyond: Unleashing Efficient Adversarial Training
Dolatabadi, Hadi M., Erfani, Sarah, Leckie, Christopher
Neural networks are vulnerable to adversarial attacks: adding well-crafted, imperceptible perturbations to their input can modify their output. Adversarial training is one of the most effective approaches in training robust models against such attacks. However, it is much slower than vanilla training of neural networks since it needs to construct adversarial examples for the entire training data at every iteration, hampering its effectiveness. Recently, Fast Adversarial Training (FAT) was proposed that can obtain robust models efficiently. However, the reasons behind its success are not fully understood, and more importantly, it can only train robust models for $\ell_\infty$-bounded attacks as it uses FGSM during training. In this paper, by leveraging the theory of coreset selection, we show how selecting a small subset of training data provides a general, more principled approach toward reducing the time complexity of robust training. Unlike existing methods, our approach can be adapted to a wide variety of training objectives, including TRADES, $\ell_p$-PGD, and Perceptual Adversarial Training (PAT). Our experimental results indicate that our approach speeds up adversarial training by 2-3 times while experiencing a slight reduction in the clean and robust accuracy.
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Zhao, Yusheng, Chen, Jinyu, Gao, Chen, Wang, Wenguan, Yang, Lirong, Ren, Haibing, Xia, Huaxia, Liu, Si
Vision-language navigation is the task of directing an embodied agent to navigate in 3D scenes with natural language instructions. For the agent, inferring the long-term navigation target from visual-linguistic clues is crucial for reliable path planning, which, however, has rarely been studied before in literature. In this article, we propose a Target-Driven Structured Transformer Planner (TD-STP) for long-horizon goal-guided and room layout-aware navigation. Specifically, we devise an Imaginary Scene Tokenization mechanism for explicit estimation of the long-term target (even located in unexplored environments). In addition, we design a Structured Transformer Planner which elegantly incorporates the explored room layout into a neural attention architecture for structured and global planning. Experimental results demonstrate that our TD-STP substantially improves previous best methods' success rate by 2% and 5% on the test set of R2R and REVERIE benchmarks, respectively. Our code is available at https://github.com/YushengZhao/TD-STP .
Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression
Konstantinov, Andrei V., Kirpichenko, Stanislav R., Utkin, Lev V.
The efficient treatment for a patient with her/his clinical and other characteristics [1, 2] can be regarded as an important goal of the real personalized medicine. The goal can be achieved by means of the machine learning methods due to the increasing amount of available electronic health records which are a basis for developing accurate models. To estimate the treatment effect, patients are divided into two groups called treatment and control, and then patients from the different groups are compared. One of the popular measures of the efficient treatment used in machine learning models is the average treatment effect (ATE) [3], which is estimated on the basis of observed data about patients as the mean difference between outcomes of patients from the treatment and control groups. Due to the difference between the patients characteristics and the difference between their responses to a particular treatment, the treatment effect is measured by the conditional average treatment effects (CATE) or the heterogeneous treatment effect (HTE) defined as ATE conditional on a patient feature vector [4, 5, 6, 7]. Two main problems can be pointed out when CATE is estimated. The first one is that the control group is usually larger than the treatment group. As a result, we meet the problem of a small training dataset, which does not allow us to apply directly many efficient machine learning methods.
A brain-computer startup beat Elon Musk's Neuralink to implanting its first device in a US patient
Synchron, a brain-computer interface startup, reportedly implanted its first device in a US patient earlier this month -- overtaking Elon Musk's Neuralink for the third time. The startup implanted a 1.5-inch device into the brain of an ALS patient at Mount Sinai West medical center in New York on July 6, Bloomberg first reported. A spokesperson from Synchron did not immediately respond to a request for comment. The purpose of the device is to allow the patient to communicate -- even after they have lost the ability to move -- by using their thoughts to send emails and texts. Bloomberg reported that Synchron has already implanted the device in four patients in Australia who have been able to use the brain implant to send messages on WhatsApp and shop online.
The Air Force plans to test an AI copilot on its cargo planes
On July 13, Boston's Merlin Labs announced that it would be working with the US Air Force to add autonomy to the C-130J Super Hercules cargo transport plane. Merlin's technology is a kind of advanced auto-copilot, designed to take over the responsibilities of one crew member in flight while being supervised by a human pilot. If the technology delivers as promised, it will allow planes that normally fly with two human pilots to operate with just one, and could even allow single-seater planes to fly fully autonomously. The same day that Merlin announced its partnership with the Air Force, it also announced a second round of $105 million in funding, which combined with a first round means the company has $130 million of runway to develop its technologies. This funding, says Merlin Labs CEO Matthew George, will help the company continue to develop "the world's most capable, safest and flexible pilot, that will eventually enable very large aircraft to fly with reduced crew and small aircraft to fly totally uncrewed."
Indoor Localization for Personalized Ambient Assisted Living of Multiple Users in Multi-Floor Smart Environments
Thakur, Nirmalya, Han, Chia Y.
This paper presents a multifunctional interdisciplinary framework that makes four scientific contributions towards the development of personalized ambient assisted living, with a specific focus to address the different and dynamic needs of the diverse aging population in the future of smart living environments. First, it presents a probabilistic reasoning-based mathematical approach to model all possible forms of user interactions for any activity arising from the user diversity of multiple users in such environments. Second, it presents a system that uses this approach with a machine learning method to model individual user profiles and user-specific user interactions for detecting the dynamic indoor location of each specific user. Third, to address the need to develop highly accurate indoor localization systems for increased trust, reliance, and seamless user acceptance, the framework introduces a novel methodology where two boosting approaches Gradient Boosting and the AdaBoost algorithm are integrated and used on a decision tree-based learning model to perform indoor localization. Fourth, the framework introduces two novel functionalities to provide semantic context to indoor localization in terms of detecting each user's floor-specific location as well as tracking whether a specific user was located inside or outside a given spatial region in a multi-floor-based indoor setting. These novel functionalities of the proposed framework were tested on a dataset of localization-related Big Data collected from 18 different users who navigated in 3 buildings consisting of 5 floors and 254 indoor spatial regions. The results show that this approach of indoor localization for personalized AAL that models each specific user always achieves higher accuracy as compared to the traditional approach of modeling an average user.