Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning. Albeit its effectiveness, the optimization of MAML can be challenging due to the innate bilevel problem structure. Specifically, the loss landscape of MAML is much more complex with possibly more saddle points and local minimizers than its empirical risk minimization counterpart. To address this challenge, we leverage the recently invented sharpness-aware minimization and develop a sharpness-aware MAML approach that we term Sharp-MAML. We empirically demonstrate that Sharp-MAML and its computation-efficient variant can outperform the plain-vanilla MAML baseline (e.g., $+3\%$ accuracy on Mini-Imagenet). We complement the empirical study with the convergence rate analysis and the generalization bound of Sharp-MAML. To the best of our knowledge, this is the first empirical and theoretical study on sharpness-aware minimization in the context of bilevel learning. The code is available at https://github.com/mominabbass/Sharp-MAML.

Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. However, modelling global correlations with multi-head self-attention (MSA) layers leads to two widely recognized issues: the massive computational resource consumption and the lack of intrinsic inductive bias for modelling local visual patterns. To solve both issues, we devise a simple yet effective method named Single-Path Vision Transformer pruning (SPViT), to efficiently and automatically compress the pre-trained ViTs into compact models with proper locality added. Specifically, we first propose a novel weight-sharing scheme between MSA and convolutional operations, delivering a single-path space to encode all candidate operations. In this way, we cast the operation search problem as finding which subset of parameters to use in each MSA layer, which significantly reduces the computational cost and optimization difficulty, and the convolution kernels can be well initialized using pre-trained MSA parameters. Relying on the single-path space, we further introduce learnable binary gates to encode the operation choices, which are jointly optimized with network parameters to automatically determine the configuration of each layer. We conduct extensive experiments on two representative ViTs showing that our SPViT achieves a new SOTA for pruning on ImageNet-1k. For example, our SPViT can trim 52.0% FLOPs for DeiT-B and get an impressive 0.6% top-1 accuracy gain simultaneously. The source code is available at https://github.com/ziplab/SPViT.

Continual learning of a stream of tasks is an active area in deep neural networks. The main challenge investigated has been the phenomenon of catastrophic forgetting or interference of newly acquired knowledge with knowledge from previous tasks. Recent work has investigated forward knowledge transfer to new tasks. Backward transfer for improving knowledge gained during previous tasks has received much less attention. There is in general limited understanding of how knowledge transfer could aid tasks learned continually. We present a theory for knowledge transfer in continual supervised learning, which considers both forward and backward transfer. We aim at understanding their impact for increasingly knowledgeable learners. We derive error bounds for each of these transfer mechanisms. These bounds are agnostic to specific implementations (e.g. deep neural networks). We demonstrate that, for a continual learner that observes related tasks, both forward and backward transfer can contribute to an increasing performance as more tasks are observed.

Thirty-five years ago, New Zealand adopted a law declaring the official status of Te Reo Māori, the language spoken by the country's indigenous Māori people. Decades of repression put the language, also called simply te reo, under serious threat: only one in four Māori spoke it by 1960, with a very low percentage of speakers among children. Since then, the language has started regaining lost ground, enjoying formally equal status with English and being taught widely to New Zealand schoolchildren. Still, reviving it as a living language takes time and persistence. Lately, the nascent te reo renaissance is gaining added momentum with the help of artificial intelligence (AI).

Of course, deep learning has made progress, but on those foundational questions, not so much; on natural language, compositionality and reasoning, which differ from the kinds of pattern recognition on which deep learning excels, these systems remain massively unreliable, exactly as you would expect from systems that rely on statistical correlations, rather than an algebra of abstraction. Minerva, the latest, greatest AI system as of this writing, with billions of "tokens" in its training, still struggles with multiplying 4-digit numbers.

There are a number of startups working to improve trash sorting with robots. AMP Robotics is near the top of the list, coupling a picker and a conveyor belt to sort materials in large, automated facilities. The technology aims to correct human shortcomings when it comes to recycling. Too often people either don't bother to separate trash, or simply don't understand where things go. Founded in 2015, CleanRobotics hopes to correct the issue at the point of disposal. The Colorado firm's flagship trashbot system uses on-board machine learning and robotic systems to sort materials from a single disposal point.

We regularly consider answering counterfactual questions in practice, such as "Would people with diabetes take a turn for the better had they choose another medication?". Observational studies are growing in significance in answering such questions due to their widespread accumulation and comparatively easier acquisition than Randomized Control Trials (RCTs). Recently, some works have introduced representation learning and domain adaptation into counterfactual inference. However, most current works focus on the setting of binary treatments. None of them considers that different treatments' sample sizes are imbalanced, especially data examples in some treatment groups are relatively limited due to inherent user preference. In this paper, we design a new algorithmic framework for counterfactual inference, which brings an idea from Meta-learning for Estimating Individual Treatment Effects (MetaITE) to fill the above research gaps, especially considering multiple imbalanced treatments. Specifically, we regard data episodes among treatment groups in counterfactual inference as meta-learning tasks. We train a meta-learner from a set of source treatment groups with sufficient samples and update the model by gradient descent with limited samples in target treatment. Moreover, we introduce two complementary losses. One is the supervised loss on multiple source treatments. The other loss which aligns latent distributions among various treatment groups is proposed to reduce the discrepancy. We perform experiments on two real-world datasets to evaluate inference accuracy and generalization ability. Experimental results demonstrate that the model MetaITE matches/outperforms state-of-the-art methods.

We introduce a new dataset for conversational question answering over Knowledge Graphs (KGs) with verbalized answers. Question answering over KGs is currently focused on answer generation for single-turn questions (KGQA) or multiple-tun conversational question answering (ConvQA). However, in a real-world scenario (e.g., voice assistants such as Siri, Alexa, and Google Assistant), users prefer verbalized answers. This paper contributes to the state-of-the-art by extending an existing ConvQA dataset with multiple paraphrased verbalized answers. We perform experiments with five sequence-to-sequence models on generating answer responses while maintaining grammatical correctness. We additionally perform an error analysis that details the rates of models' mispredictions in specified categories. Our proposed dataset extended with answer verbalization is publicly available with detailed documentation on its usage for wider utility.

DALL-E 2, an image-generating artificial intelligence (AI) has captured the public's attention with stunning portrayals of Godzilla-eating Tokyo and photorealistic images of astronauts riding horses in space. The model is the newest iteration of a text-to-image algorithm, an AI model that can generate images based on text descriptions. OpenAI, the company behind DALL-E 2, used a language model, GPT-3, and a computer vision model, CLIP, to train DALL-E 2 using 650 million images with associated text captions. The integration of these two models made it possible for OpenAI to train DALL-E 2 to generate a vast array of images in many different styles. Despite DALL-E 2's impressive accomplishments, there are significant issues with how the model portrays people and how it has acquired biases from the data it was trained on.

In a classic case of finding a balance between costs and benefits of science, researchers are grappling with the question of how artificial intelligence in medicine can and should be applied to clinical patient care – despite knowing that there are examples where it puts patients' lives at risk. The question was central to a recent university of Adelaide seminar, part of the Research Tuesdays lecture series, titled "Antidote AI." As artificial intelligence grows in sophistication and usefulness, we have begun to see it appearing more and more in everyday life. From AI traffic control and ecological studies, to machine learning finding the origins of a Martian meteorite and reading Arnhem Land rock art, the possibilities seem endless for AI research. The genuine excitement clinicians and artificial intelligence researchers feel for the prospect of AI assisting in patient care is palpable and honourable. Medicine is, after all, about helping people and the ethical foundation is "do no harm."