While a diverse collection of continual learning (CL) methods has been proposed to prevent catastrophic forgetting, a thorough investigation of their effectiveness for processing sequential data with recurrent neural networks (RNNs) is lacking. Here, we provide the first comprehensive evaluation of established CL methods on a variety of sequential data benchmarks. Specifically, we shed light on the particularities that arise when applying weight-importance methods, such as elastic weight consolidation, to RNNs. In contrast to feedforward networks, RNNs iteratively reuse a shared set of weights and require working memory to process input samples. We show that the performance of weight-importance methods is not directly affected by the length of the processed sequences, but rather by high working memory requirements, which lead to an increased need for stability at the cost of decreased plasticity for learning subsequent tasks. We additionally provide theoretical arguments supporting this interpretation by studying linear RNNs. Our study shows that established CL methods can be successfully ported to the recurrent case, and that a recent regularization approach based on hypernetworks outperforms weight-importance methods, thus emerging as a promising candidate for CL in RNNs. Overall, we provide insights on the differences between CL in feedforward networks and RNNs, while guiding towards effective solutions to tackle CL on sequential data.

Classic decision tree learning is a binary classification algorithm that constructs models with first-class transparency - every classification has a directly derivable explanation. However, learning decision trees on modern datasets generates large trees, which in turn generate decision paths of excessive depth, obscuring the explanation of classifications. To improve the comprehensibility of classifications, we propose a new decision tree model that we call Cascading Decision Trees. Cascading Decision Trees shorten the size of explanations of classifications, without sacrificing model performance overall. Our key insight is to separate the notion of a decision path and an explanation path. Utilizing this insight, instead of having one monolithic decision tree, we build several smaller decision subtrees and cascade them in sequence. Our cascading decision subtrees are designed to specifically target explanations for positive classifications. This way each subtree identifies the smallest set of features that can classify as many positive samples as possible, without misclassifying any negative samples. Applying cascading decision trees to new samples results in a significantly shorter and succinct explanation, if one of the subtrees detects a positive classification. In that case, we immediately stop and report the decision path of only the current subtree to the user as an explanation for the classification. We evaluate our algorithm on standard datasets, as well as new real-world applications and find that our model shortens the explanation depth by over 40.8% for positive classifications compared to the classic decision tree model.

Building destruction during war is a specific form of violence which is particularly harmful to civilians, commonly used to displace populations, and therefore warrants special attention. Yet, data from war-ridden areas are typically scarce, often incomplete and highly contested, when available. The lack of such data from conflict zones severely limits media reporting, humanitarian relief efforts, human rights monitoring, reconstruction initiatives, as well as the study of violent conflict in academic research. One approach has been to use remote sensing to identify destruction in satellite images[1]. This approach is gaining momentum as high-resolution imagery is becoming readily available and is updated ever quicker yielding weekly or even daily frequency. At the same time recent methodological advances related to deep learning have provided sophisticated tools to extract data from these images [2, 3, 4, 5].

Artificial intelligence promises to expertly diagnose disease in medical images and scans. However, a close look at the data used to train algorithms for diagnosing eye conditions suggests these powerful new tools may perpetuate health inequalities. A team of researchers in the UK analyzed 94 data sets--with more than 500,000 images--commonly used to train AI algorithms to spot eye diseases. They found that almost all of the data came from patients in North America, Europe, and China. Just four data sets came from South Asia, two from South America, and one from Africa; none came from Oceania.

In 2018, Alphabet's X lab said it was in the process of exploring how it could use artificial intelligence to improve farming. On Monday, X announced that its "computational agriculture" project is called Mineral. The Mineral team has spent the last several years "developing and testing a range of software and hardware prototypes based on breakthroughs in artificial intelligence, simulation, sensors, robotics and more." One of the tools that has come out of the project is a robotic plant buggy. Powered by solar panels, the machine makes its way across a farmer's field, examining every plant it passes along the way with an array of cameras and sensors.

The world's small-scale farmers now can see a path to solving global hunger over the next decade, with solutions – such as adopting climate-resilient crops through improving extension services – all culled rapidly via artificial intelligence from more than 500,000 scientific research articles. The results are synthesized in 10 new research papers – authored by 77 scientists, researchers and librarians in 23 countries – as part of Ceres2030: Sustainable Solutions to End Hunger. The project is headquartered at Cornell, with partners from the International Food Policy Research Institute (IFPRI) and the International Institute for Sustainable Development (IISD). The papers were published concurrently on Oct. 12 in four journals – Nature Plants, Nature Sustainability, Nature Machine Intelligence and Nature Food – and assembled in a comprehensive package online: Sustainable Solutions to End Hunger. Ceres2030 employed machine learning, librarian savvy and research synthesis methods to quickly scan a trove of thousands of scientific journals for ideas and websites from more than 60 agencies that can help eradicate world hunger.

Hitachi Construction Machinery (HCM) and its consolidated subsidiary, Wenco International Mining Systems, have jointly developed "ConSite Mine", a new technology platform that helps resolve problems at mine sites by remotely monitoring mining machines on a 24/7 basis through the use of IoT and AI based analysis of equipment operations data. According to Hitachi, it has developed this technology to help customers and HCM dealers predict costly maintenance issues before they occur, such as the occurrence of cracks in excavator booms or arms, by utilising machine learning and applied analysis technologies. Detailed information from these predictive alerts are provided on the web-based ConSite Mine dashboard and other items. Currently, Hitachi is piloting the technology in Australia, Zambia and Indonesia. "ConSite Mine" will be further modified based on customer feedback before wider commercial release in 2021.

How globally diverse are AI Ethics boards? An article published by MIT shows that Americans and Europeans occupy most seats on these boards. If these compositions remain like this, we run the risk that bias in AI will perpetuate. An excellent example of different views can be seen in the question of what intelligence is. While we can't scientifically explain what intelligence is, there are vastly different perceptions of how intelligent behavior expresses itself.

We present the Zero Resource Speech Challenge 2020, which aims at learning speech representations from raw audio signals without any labels. It combines the data sets and metrics from two previous benchmarks (2017 and 2019) and features two tasks which tap into two levels of speech representation. The first task is to discover low bit-rate subword representations that optimize the quality of speech synthesis; the second one is to discover word-like units from unsegmented raw speech. We present the results of the twenty submitted models and discuss the implications of the main findings for unsupervised speech learning.

Stochastic gradient descent (SGD) and its variants are mainstream methods to train deep neural networks. Since neural networks are non-convex, more and more works study the dynamic behavior of SGD and the impact to its generalization, especially the escaping efficiency from local minima. However, these works take the over-simplified assumption that the covariance of the noise in SGD is (or can be upper bounded by) constant, although it is actually state-dependent. In this work, we conduct a formal study on the dynamic behavior of SGD with state-dependent noise. Specifically, we show that the covariance of the noise of SGD in the local region of the local minima is a quadratic function of the state. Thus, we propose a novel power-law dynamic with state-dependent diffusion to approximate the dynamic of SGD. We prove that, power-law dynamic can escape from sharp minima exponentially faster than flat minima, while the previous dynamics can only escape sharp minima polynomially faster than flat minima. Our experiments well verified our theoretical results. Inspired by our theory, we propose to add additional state-dependent noise into (large-batch) SGD to further improve its generalization ability. Experiments verify that our method is effective.