Medical term normalization consists in mapping a piece of text to a large number of output classes. Given the small size of the annotated datasets and the extremely long tail distribution of the concepts, it is of utmost importance to develop models that are capable to generalize to scarce or unseen concepts. An important attribute of most target ontologies is their hierarchical structure. In this paper we introduce a simple and effective learning strategy that leverages such information to enhance the generalizability of both discriminative and generative models. The evaluation shows that the proposed strategy produces state-of-the-art performance on seen concepts and consistent improvements on unseen ones, allowing also for efficient zero-shot knowledge transfer across text typologies and datasets.

Program synthesis or code generation aims to generate a program that satisfies a problem specification. Recent approaches using large-scale pretrained language models (LMs) have shown promising results, yet they have some critical limitations. In particular, they often follow a standard supervised fine-tuning procedure to train a code generation model only from the pairs of natural-language problem descriptions and ground-truth programs. Such paradigm largely ignores some important but potentially useful signals in the problem specification such as unit tests, which thus often results in poor performance when solving complex unseen coding tasks. To address the limitations, we propose "CodeRL", a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL). Specifically, during training, we treat the code-generating LM as an actor network, and introduce a critic network that is trained to predict the functional correctness of generated programs and provide dense feedback signals to the actor. During inference, we introduce a new generation procedure with a critical sampling strategy that allows a model to automatically regenerate programs based on feedback from example unit tests and critic scores. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives, larger model sizes, and better pretraining data. Our method not only achieves new SOTA results on the challenging APPS benchmark, but also shows strong zero-shot transfer capability with new SOTA results on the simpler MBPP benchmark.

Music is one of the Gardner's intelligences in his theory of multiple intelligences. How humans perceive and understand music is still being studied and is crucial to develop artificial intelligence models that imitate such processes. Music generation with Artificial Intelligence is an emerging field that is gaining much attention in the recent years. In this paper, we describe how humans compose music and how new AI systems could imitate such process by comparing past and recent advances in the field with music composition techniques. To understand how AI models and algorithms generate music and the potential applications that might appear in the future, we explore, analyze and describe the agents that take part of the music generation process: the datasets, models, interfaces, the users and the generated music. We mention possible applications that might benefit from this field and we also propose new trends and future research directions that could be explored in the future.

For the majority of the machine learning community, the expensive nature of collecting high-quality human-annotated data and the inability to efficiently finetune very large state-of-the-art pretrained models on limited compute are major bottlenecks for building models for new tasks. We propose a zero-shot simple approach for one such task, Video Moment Retrieval (VMR), that does not perform any additional finetuning and simply repurposes off-the-shelf models trained on other tasks. Our three-step approach consists of moment proposal, moment-query matching and postprocessing, all using only off-the-shelf models. On the QVHighlights benchmark for VMR, we vastly improve performance of previous zero-shot approaches by at least 2.5x on all metrics and reduce the gap between zero-shot and state-of-the-art supervised by over 74%. Further, we also show that our zero-shot approach beats non-pretrained supervised models on the Recall metrics and comes very close on mAP metrics; and that it also performs better than the best pretrained supervised model on shorter moments. Finally, we ablate and analyze our results and propose interesting future directions.

OpenAI, the San Francisco-based lab behind AI systems like GPT-3 and DALL-E 2, today launched a new program to provide early-stage AI startups with capital and access to OpenAI tech and resources. Called Converge, the cohort will be financed by the OpenAI Startup Fund, OpenAI says. The $100 million entrepreneurial tranche was announced last May and was backed by Microsoft and other partners. The 10 or so founders chosen for Converge will receive $1 million each and admission to five weeks of office hours, workshops and events with OpenAI staff, as well as early access to OpenAI models and "programming tailored to AI companies." "We're excited to meet groups across all phases of the seed stage, from pre-idea solo founders to co-founding teams already working on a product," OpenAI writes in a blog post shared with TechCrunch ahead of today's announcement.

By having a single model that is exposed to and trained on many different languages, we get much better performance on our low resource languages,

The ESM Metagenomic Atlas database contains structure predictions for 617 million proteins.Credit: ESM Metagenomic Atlas (CC BY 4.0) When London-based Deep Mind unveiled predicted structures for some 220 million proteins this year, it covered nearly every protein from known organisms in DNA databases. Now, another tech giant is filling in the dark matter of our protein universe. Researchers at Meta (formerly Facebook, headquartered in Menlo Park, California) have used artificial intelligence (AI) to predict the structures of some 600 million proteins from bacteria, viruses and other microbes that haven't been characterized. 'It will change everything': DeepMind's AI makes gigantic leap in solving protein structures "These are the structures we know the least about. These are incredibly mysterious proteins. I think they offer the potential for great insight into biology," says Alexander Rives, the research lead for Meta AI's protein team.

AI researchers at Meta say they have developed the largest protein-folding model of its kind to date, and that it is capable of predicting the structure of more than 600 million proteins.… The team released the 15-billion-parameter ESM-2 transformer-based model and a database of its protein structure predictions, dubbed the ESM Metagenomic Atlas, on Tuesday. This database includes protein shapes that haven't been observed yet by scientists. Proteins are complex biological molecules containing up of 20 types of amino acids, and perform all sorts of biological functions in organisms. Crucially, they fold up into intricate 3D structures, the shape of which is vital to how they operate; knowing their shape helps scientists understand how they function, and from that, helps them figure out ways to mimic, alter, or counter that behavior. Unfortunately, you can't just take the amino acid formula and immediately work out the eventual structure.

Researchers from DeepMind, the UK's juggernaut AI lab, have forsaken the noble games of chess and Go for a more plebeian delight: football. The Google sister company yesterday published a research paper and accompanying blog post detailing its new neural probabilistic motor primitives (NPMP) -- a method by which artificial intelligence agents can learn to operate physical bodies. An NPMP is a general-purpose motor control module that translates short-horizon motor intentions to low-level control signals, and it's trained offline or via RL by imitating motion capture (MoCap) data, recorded with trackers on humans or animals performing motions of interest. Up front: Essentially, the DeepMind team created an AI system that can learn how to do things inside of a physics simulator by watching videos of other agents performing those tasks. And, of course, if you've got a giant physics engine and an endless supply of curious robots, the only rational thing to do is to teach it how to dribble and shoot: We optimized teams of agents to play simulated football via reinforcement learning, constraining the solution space to that of plausible movements learned using human motion capture data. Background: In order to train AI to operate and control robots in the world, researchers have to prepare the machines for reality.

Analysis While the legal and ethical implications of assistive AI models like GitHub's Copilot continue to be sorted out, computer scientists continue to find uses for large language models and urge educators to adapt. Brett A. Becker, assistant professor at University College Dublin in Ireland, provided The Register with pre-publication copies of two research papers exploring the educational risks and opportunities of AI tools for generating programming code. The papers have been accepted at the 2023 SIGCSE Technical Symposium on Computer Science Education, to be held March 15 to 18 in Toronto, Canada. In June, GitHub Copilot, a machine learning tool that automatically suggests programming code in response to contextual prompts, emerged from a year long technical preview, just as concerns about the way its OpenAI Codex model was trained and the implications of AI models for society coalesced into focused opposition. In "Programming Is Hard – Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation" [PDF], Becker and co-authors Paul Denny (University of Auckland, New Zealand), James Finnie-Ansley (University of Auckland), Andrew Luxton-Reilly (University of Auckland), James Prather (Abilene Christian University, USA), and Eddie Antonio Santos (University College Dublin) argue that the educational community needs to deal with the immediate opportunities and challenges presented by AI-driven code generation tools.