Oceania
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
Iyer, Abhiram, Grewal, Karan, Velu, Akash, Souza, Lucas Oliveira, Forest, Jeremy, Ahmad, Subutai
A key challenge for AI is to build embodied systems that operate in dynamically changing environments. Such systems must adapt to changing task contexts and learn continuously. Although standard deep learning systems achieve state of the art results on static benchmarks, they often struggle in dynamic scenarios. In these settings, error signals from multiple contexts can interfere with one another, ultimately leading to a phenomenon known as catastrophic forgetting. In this article we investigate biologically inspired architectures as solutions to these problems. Specifically, we show that the biophysical properties of dendrites and local inhibitory systems enable networks to dynamically restrict and route information in a context-specific manner. Our key contributions are as follows. First, we propose a novel artificial neural network architecture that incorporates active dendrites and sparse representations into the standard deep learning framework. Next, we study the performance of this architecture on two separate benchmarks requiring task-based adaptation: Meta-World, a multi-task reinforcement learning environment where a robotic agent must learn to solve a variety of manipulation tasks simultaneously; and a continual learning benchmark in which the model's prediction task changes throughout training. Analysis on both benchmarks demonstrates the emergence of overlapping but distinct and sparse subnetworks, allowing the system to fluidly learn multiple tasks with minimal forgetting. Our neural implementation marks the first time a single architecture has achieved competitive results on both multi-task and continual learning settings. Our research sheds light on how biological properties of neurons can inform deep learning systems to address dynamic scenarios that are typically impossible for traditional ANNs to solve.
Binary Diffing as a Network Alignment Problem via Belief Propagation
In this paper, we address the problem of finding a correspondence, or matching, between the functions of two programs in binary form, which is one of the most common task in binary diffing. We introduce a new formulation of this problem as a particular instance of a graph edit problem over the call graphs of the programs. In this formulation, the quality of a mapping is evaluated simultaneously with respect to both function content and call graph similarities. We show that this formulation is equivalent to a network alignment problem. We propose a solving strategy for this problem based on max-product belief propagation. Finally, we implement a prototype of our method, called QBinDiff, and propose an extensive evaluation which shows that our approach outperforms state of the art diffing tools.
Exercise 'sweet spot' can reverse cognitive decline, study suggests
'Exercise can improve hippocampal function, however, the amount of exercise and mechanisms mediating improvement remain largely unknown.' The active place avoidance (APA) task is a dry-arena task used to assess spatial navigation and memory in rodents. In this task, a subject is put on a rotating circular arena and avoids an invisible sector that is stable in relation to the room. Rotation of the arena means that the subject's avoidance must be active, otherwise the subject will be moved in the to-be-avoided sector by the rotation of the arena and a slight electric shock will be administered.
iMediSync Unveils World's First Integrated Wireless Brain EEG At CES 2022
They are launching their first therapeutic device – iSyncWave, which integrates both EEG brain mapping and LED therapy at 2022 CES Las Vegas. An open-to-public CES Show Day press conference will take place on Jan 6th at the CES provided conference room. As the company slogan states, "Overcome mental pandemic in just 10 minutes, "press and visitors can expect a complete analysis report of EEG (brainwave) and HRV (Heart rate variability) operated by AI deep learning algorithms in just 10 minutes to assess the condition of brain and dysfunctionality including the early diagnostic insights which could play a major role in discovering a neuro-related diseases in early stages. Recently with the addition of software upgrades, and an FCC approval allowed the iSyncWave to be exported to the EU, Asia, and Australia already use for research and diagnostics. "We will provide life-changing mental healthcare experience to introduce neuro-mental illness biomarker and the patent EEG/HRV technology," says CEO Dr. Seung Wan Kang.
Read All of the Mind-Blowing Sci-Fi Stories We Published This Year
This year at Future Tense Fiction we've spent a lot of time thinking about how, in many ways, 2021 has felt a lot like 2020. But at the same time, so much has changed--how we work and think, how we commute, how we interact with animals, technology, and our fellow humans. This year we published 11 stories (we took December off!) that touch upon relationships, transportation, right to repair and supply chain shortages, communication, information overload and scarcity, and much, much more. We broadly explored themes like learning futures, with Simon Brown's "Speaker" (where humans learn to communicate with other species and struggle to overcome the assumption of human excellence), Leigh Alexander's "The Void" (about the struggle with information scarcity in an information-overloaded world) and Shiv Ramdas' "The Trolley Solution" (about a university attempting to automate how it teaches its students), as well as ideas of mobility--a theme we're continuing into 2022, so stay tuned--with Linda Nagata's "Ride" (about a neighborhood that's embraced an algorithm to run all of its traffic and transit patterns). We began publishing fiction back in 2016 and made it monthly as of January 2018.
Applying data science in the life insurance industry -- a perspective from a qualified actuary
To summarise, this use case presents a way for actuaries to automatically classify free-text claims causes data into pre-defined categories for further analyses. Ultimately, with the help of BERT, computers are able to understand human language. For this instance, computers are able to understand and compare medical terms or description of a claims event, which can be messy at times. The alternative which is manual filtering in Excel is not practical, especially for large number of claims. As mentioned previously, Excel has been the primary ETL tool for most life insurance actuaries.
Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction
Liu, Zhenguang, Wu, Shuang, Jin, Shuyuan, Ji, Shouling, Liu, Qi, Lu, Shijian, Cheng, Li
Predicting human motion from historical pose sequence is crucial for a machine to succeed in intelligent interactions with humans. One aspect that has been obviated so far, is the fact that how we represent the skeletal pose has a critical impact on the prediction results. Yet there is no effort that investigates across different pose representation schemes. We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. Moreover, recent approaches build upon off-the-shelf RNN units for motion prediction. These approaches process input pose sequence sequentially and inherently have difficulties in capturing long-term dependencies. In this paper, we propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction which simultaneously models local motion contexts and a global context. We further explore a geodesic loss and a forward kinematics loss for the motion prediction task, which have more geometric significance than the widely employed L2 loss. Interestingly, we applied our method to a range of articulate objects including human, fish, and mouse. Empirical results show that our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency, such as retaining natural human-like motions over 50 seconds predictions. Our codes are released.
Random cohort effects and age groups dependency structure for mortality modelling and forecasting: Mixed-effects time-series model approach
There have been significant efforts devoted to solving the longevity risk given that a continuous growth in population ageing has become a severe issue for many developed countries over the past few decades. The Cairns-Blake-Dowd (CBD) model, which incorporates cohort effects parameters in its parsimonious design, is one of the most well-known approaches for mortality modelling at higher ages and longevity risk. This article proposes a novel mixed-effects time-series approach for mortality modelling and forecasting with considerations of age groups dependence and random cohort effects parameters. The proposed model can disclose more mortality data information and provide a natural quantification of the model parameters uncertainties with no pre-specified constraint required for estimating the cohort effects parameters. The abilities of the proposed approach are demonstrated through two applications with empirical male and female mortality data. The proposed approach shows remarkable improvements in terms of forecast accuracy compared to the CBD model in the short-, mid-and long-term forecasting using mortality data of several developed countries in the numerical examples.
SimSR: Simple Distance-based State Representation for Deep Reinforcement Learning
Zang, Hongyu, Li, Xin, Wang, Mingzhong
This work explores how to learn robust and generalizable state representation from image-based observations with deep reinforcement learning methods. Addressing the computational complexity, stringent assumptions, and representation collapse challenges in the existing work of bisimulation metric, we devise Simple State Representation (SimSR) operator, which achieves equivalent functionality while reducing the complexity by an order in comparison with bisimulation metric. SimSR enables us to design a stochastic-approximation-based method that can practically learn the mapping functions (encoders) from observations to latent representation space. Besides the theoretical analysis, we experimented and compared our work with recent state-of-the-art solutions in visual MuJoCo tasks. The results show that our model generally achieves better performance and has better robustness and good generalization.
Does QA-based intermediate training help fine-tuning language models for text classification?
Fine-tuning pre-trained language models for downstream tasks has become a norm for NLP. Recently it is found that intermediate training based on high-level inference tasks such as Question Answering (QA) can improve the performance of some language models for target tasks. However it is not clear if intermediate training generally benefits various language models. In this paper, using the SQuAD-2.0 QA task for intermediate training for target text classification tasks, we experimented on eight tasks for single-sequence classification and eight tasks for sequence-pair classification using two base and two compact language models. Our experiments show that QA-based intermediate training generates varying transfer performance across different language models, except for similar QA tasks.