Oceania
Disruptive Ideas that are going to shape and change the NZ landscape (via Passle)
Our friends over at the Spinoff recently reported that what is predicted to be a driverless future is not all that far off for big little New Zealand. Auckland-based company HMI Technologies who are the people behind Ohmio, have produced self-driving shuttles which are in store for Christchurch Airport in late 2018. The shuttles are electrically powered and operate using self-mapping artificial intelligence. As the shuttle begins to drive it's specified route means, the system will record the coordinates and speed embedding that into how the device is controlled. The shuttle can then replicate that journey and have different routes loaded onto it as needed.
Meet the TrashBot: CleanRobotics is using machine learning to keep recycling from going to waste
At a mall in Sydney, Australia, "the world's first smart trash can" is fastidiously photographing, weighing, and sorting garbage. The industrious TrashBot is a long way from home. Trashbot was born in Pittsburgh at the AlphaLab Gear startup accelerator. There, the CleanRobotics team has been developing a machine that uses cameras, sensors, and machine learning to ensure that garbage ends up in the landfill and recyclables don't. They're tackling a problem that most environmentalists would agree needs to be solved: only about 20 percent of what goes in those blue bins actually ends up recycled, according to CleanRobotics co-founder Tanner Cook.
Aeolus Robotics: This robot will bring you beer
The newest artificial intelligence out of San Francisco-based tech firm Aeolus Robotics is a revolutionary robot that can pretty much second guess your family's movements, their identity, and even perform household duties from moping the floor to getting drinks out of the fridge. While the flying cars that were promised to us in The Jetsons are lamentably still on the drawing board, a digital domestic goddess by the likes of Rosie The Robot has actually arrived. The mechanical mate, which is expected to be available by the end of the year, will reportedly cost as much as a car at around $US20,000 ($25,000) and is described by Aeolus as being the height and weight of a 12 year old -- however undoubtedly more house-trained. Just like a creepy scene out of (insert favourite Sci Fi film here) this amazing android can distinguish between family members' faces, recognise where household items are supposed to go (then put them back in place) and can keep a sly eye open for emergencies like a fire or notice a change in posture and possibly prevent a fall. The yet to be officially named "Aeolus Robot" can also move furniture, find lost items and even learn the household schedule via an information sharing network.
Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning
Fruit, Ronan, Pirotta, Matteo, Lazaric, Alessandro, Ortner, Ronald
We introduce SCAL, an algorithm designed to perform efficient exploration-exploitation in any unknown weakly-communicating Markov Decision Process (MDP) for which an upper bound c on the span of the optimal bias function is known. For an MDP with S states, A actions and Gamma <= S possible next states, we prove a regret bound of O(c\sqrt{Gamma SAT}), which significantly improves over existing algorithms (e.g., UCRL and PSRL), whose regret scales linearly with the MDP diameter D. In fact, the optimal bias span is finite and often much smaller than D (e.g., D=infinity in non-communicating MDPs). A similar result was originally derived by Bartlett and Tewari (2009) for REGAL.C, for which no tractable algorithm is available. In this paper, we relax the optimization problem at the core of REGAL.C, we carefully analyze its properties, and we provide the first computationally efficient algorithm to solve it. Finally, we report numerical simulations supporting our theoretical findings and showing how SCAL significantly outperforms UCRL in MDPs with large diameter and small span.
Efficient Empirical Risk Minimization with Smooth Loss Functions in Non-interactive Local Differential Privacy
Wang, Di, Gaboardi, Marco, Xu, Jinhui
In this paper, we study the Empirical Risk Minimization problem in the non-interactive local model of differential privacy. We first show that if the ERM loss function is $(\infty, T)$-smooth, then we can avoid a dependence of the sample complexity, to achieve error $\alpha$, on the exponential of the dimensionality $p$ with base $1/\alpha$ ({\em i.e.,} $\alpha^{-p}$), which answers a question in \cite{smith2017interaction}. Our approach is based on Bernstein polynomial approximation. Then, we propose player-efficient algorithms with $1$-bit communication complexity and $O(1)$ computation cost for each player. The error bound is asymptotically the same as the original one. Also with additional assumptions we show a server efficient algorithm with polynomial running time. At last, we propose (efficient) non-interactive locally differential private algorithms, based on different types of polynomial approximations, for learning the set of k-way marginal queries and the set of smooth queries.
Predicting Adversarial Examples with High Confidence
Galloway, Angus, Taylor, Graham W., Moussa, Medhat
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to adversarial examples. This work is one of the most proactive approaches taken to date, as we link robustness with non-calibrated model confidence on noisy images, providing a data-augmentation-free path forward. The adversarial examples phenomenon is most easily explained by the trend of increasing non-regularized model capacity, while the diversity and number of samples in common datasets has remained flat. Test accuracy has incorrectly been associated with true generalization performance, ignoring that training and test splits are often extremely similar in terms of the overall representation space. The transferability property of adversarial examples was previously used as evidence against overfitting arguments, a perceived random effect, but overfitting is not always random.
Adversarially Regularized Graph Autoencoder
Pan, Shirui, Hu, Ruiqi, Long, Guodong, Jiang, Jing, Yao, Lina, Zhang, Chengqi
Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics. Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data, but they have mostly ignored the data distribution of the latent codes from the graphs, which often results in inferior embedding in real-world graph data. In this paper, we propose a novel adversarial graph embedding framework for graph data. The framework encodes the topological structure and node content in a graph to a compact representation, on which a decoder is trained to reconstruct the graph structure. Furthermore, the latent representation is enforced to match a prior distribution via an adversarial training scheme. To learn a robust embedding, two variants of adversarial approaches, adversarially regularized graph autoencoder (ARGA) and adversarially regularized variational graph autoencoder (ARVGA), are developed. Experimental studies on real-world graphs validate our design and demonstrate that our algorithms outperform baselines by a wide margin in link prediction, graph clustering, and graph visualization tasks.
Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
Atkinson, Craig, McCane, Brendan, Szymanski, Lech, Robins, Anthony
In general, neural networks are not currently capable of learning tasks in a sequential fashion. When a novel, unrelated task is learnt by a neural network, it substantially forgets how to solve previously learnt tasks. One of the original solutions to this problem is pseudo-rehearsal, which involves learning the new task while rehearsing generated items representative of the previous task/s. This is very effective for simple tasks. However, pseudo-rehearsal has not yet been successfully applied to very complex tasks because in these tasks it is difficult to generate representative items. We accomplish pseudo-rehearsal by using a Generative Adversarial Network to generate items so that our deep network can learn to sequentially classify the CIFAR-10, SVHN and MNIST datasets. After training on all tasks, our network loses only 1.67% absolute accuracy on CIFAR-10 and gains 0.24% absolute accuracy on SVHN. Our model's performance is a substantial improvement compared to the current state of the art solution.
Dual Memory Neural Computer for Asynchronous Two-view Sequential Learning
Le, Hung, Tran, Truyen, Venkatesh, Svetha
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that aims to model these complex interactions between two asynchronous sequential views. Our model uses two encoders for reading from and writing to two external memories for encoding input views. The intra-view interactions and the long-term dependencies are captured by the use of memories during this encoding process. There are two modes of memory accessing in our system: late-fusion and early-fusion, corresponding to late and early inter-view interactions. In the late-fusion mode, the two memories are separated, containing only view-specific contents. In the early-fusion mode, the two memories share the same addressing space, allowing cross-memory accessing. In both cases, the knowledge from the memories will be combined by a decoder to make predictions over the output space. The resulting dual memory neural computer is demonstrated on a comprehensive set of experiments, including a synthetic task of summing two sequences and the tasks of drug prescription and disease progression in healthcare. The results demonstrate competitive performance over both traditional algorithms and deep learning methods designed for multi-view problems.
Relational Autoencoder for Feature Extraction
Meng, Qinxue, Catchpoole, Daniel, Skillicorn, David, Kennedy, Paul J.
Feature extraction becomes increasingly important as data grows high dimensional. Autoencoder as a neural network based feature extraction method achieves great success in generating abstract features of high dimensional data. However, it fails to consider the relationships of data samples which may affect experimental results of using original and new features. In this paper, we propose a Relation Autoencoder model considering both data features and their relationships. We also extend it to work with other major autoencoder models including Sparse Autoencoder, Denoising Autoencoder and Variational Autoencoder. The proposed relational autoencoder models are evaluated on a set of benchmark datasets and the experimental results show that considering data relationships can generate more robust features which achieve lower construction loss and then lower error rate in further classification compared to the other variants of autoencoders.