In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models and can even learn general task-agnostic representations for efficient finetuning to downstream tasks. However, deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. This paper provides an overview of model reprogramming to bridge this gap. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing and reusing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning, where the source and target domains can be vastly different. In many applications, model reprogramming outperforms transfer learning and training from scratch. This paper elucidates the methodology of model reprogramming, summarizes existing use cases, provides a theoretical explanation of the success of model reprogramming, and concludes with a discussion on open-ended research questions and opportunities. A list of model reprogramming studies is actively maintained and updated at https://github.com/IBM/model-reprogramming.

In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement that, for the first time, empowers model reprogramming on ASR. Specifically, we investigate how to select trainable components (i.e., encoder) of a conformer-based RNN-Transducer, as a frozen pre-trained backbone. Experiments on a seven-language multilingual LibriSpeech speech (MLS) task show that model reprogramming only requires 4.2% (11M out of 270M) to 6.8% (45M out of 660M) of its original trainable parameters from a full ASR model to perform competitive results in a range of 11.9% to 8.1% WER averaged across different languages. In addition, we discover different setups to make large-scale pre-trained ASR succeed in both monolingual and multilingual speech recognition. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses (e.g., w2v-bert) in terms of lower WER and better training efficiency.

The performance of approaches to Music Instrument Classification, a popular task in Music Information Retrieval, is often impacted and limited by the lack of availability of annotated data for training. We propose to address this issue with "reprogramming," a technique that utilizes pre-trained deep and complex neural networks originally targeting a different task by modifying and mapping both the input and output of the pre-trained model. We demonstrate that reprogramming can effectively leverage the power of the representation learned for a different task and that the resulting reprogrammed system can perform on par or even outperform state-of-the-art systems at a fraction of training parameters. Our results, therefore, indicate that reprogramming is a promising technique potentially applicable to other tasks impeded by data scarcity.

How will pedestrians fare in an era of self-driving cars? When growing up, most children are taught the rather simple but altogether life-saving idea that they should look both ways before crossing the street. Why is there a need to teach such a safety precaution? Because the streets are occupied by moving objects, including heavy ones that can knock the stuffing out of you. Cars coming down a street can ram into a person and the result is downright ugly. It is likely that the pedestrian struck by a car is going to suffer some number of injuries, ranging from mild scratches and a few broken bones to the sad and all too often loss-of-life entirely. According to U.S. government statistics, the year 2019 had about 6,200 pedestrian fatalities and approximately 76,000 pedestrians were injured, which is generally the annual counts that occur year after year (for my collection of driving stats, see the link here).

Dan Tepfer is an acclaimed jazz pianist and composer who has played venues from Tokyo's Sumida Triphony Hall to New York's Village Vanguard. He also has a degree in astrophysics and writes computer programs. Born to a mother who sang in the Paris Opera and a plant-geneticist father who brought a Macintosh Plus home in the 1980s, Tepfer sees the worlds of art and science as entirely complementary. In his latest project, Acoustic Informatics, Tepfer uses a player piano, the automated instrument that occasionally appears in airports and Wild West saloons. Next month, he will present his first concert in New York City -- where he's lived for more than a decade -- to showcase this project at the Jazz Gallery, a venue known for its experimentation.

Several weeks ago, artist and coder Janelle Shane tried to train a neural network to name paint colors. "Stanky Bean" was a kind of dull pink, and "Stoner Blue" was gray. Then there were the three shades of brown known as "Dope," "Burble Simp," and "Turdly." First, Shane realized that part of the initial problem was that she'd cranked up the neural net's "temperature" variable, which meant that it was picking less likely (or "more creative") possibilities as it generated paint names letter-by-letter. So she turned the temperature variable down, and found that the names were still pretty silly but they at least matched the colors most of the time.

"Exponential data problems is very challenging and it's why it's hard to apply machine learning to genomics" Last week at the Deep Learning Summit in San Francisco we had lots of great speakers including Brendan Frey, Co-Founder & CEO at Deep Genomics; Andrew Tulloch, Research Engineer at Facebook and Andrej Karpathy, Research Scientist at OpenAI, amongst many others. Incase you missed the presentation from Brendan Frey from Deep Genomics, we are sharing with you the full recording of the video below! Deep Genomics bring together machine learning and experimental biology. Their systems "predict the molecular effect of genetic variation, opening a new and exciting path to discovery for disease diagnostics and therapies." Brenden talks about the recently developed gene editing systems that has made it possible to edit our genomes.