Executives at artificial intelligence companies may like to tell us that AGI is almost here, but the latest models still need some additional tutoring to help them be as clever as they can. Scale AI, a company that's played a key role in helping frontier AI firms build advanced models, has developed a platform that can automatically test a model across thousands of benchmarks and tasks, pinpoint weaknesses, and flag additional training data that ought to help enhance their skills. Scale, of course, will supply the data required. Scale rose to prominence providing human labor for training and testing advanced AI models. Large language models (LLMs) are trained on oodles of text scraped from books, the web, and other sources.

Databricks, a company that helps big businesses build custom artificial intelligence models, has developed a machine learning trick that can boost the performance of an AI model without the need for clean labelled data. Jonathan Frankle, chief AI scientist at Databricks, spent the past year talking to customers about the key challenges they face in getting AI to work reliably. The problem, Frankle says, is dirty data. "Everybody has some data, and has an idea of what they want to do," Frankle says. But the lack of clean data makes it challenging to fine-tune a model to perform a specific task.. "Nobody shows up with nice, clean fine-tuning data that you can stick into a prompt or an [application programming interface]," for a model.

This past Monday, about a dozen engineers and executives at data science and AI company Databricks gathered in conference rooms connected via Zoom to learn if they had succeeded in building a top artificial intelligence language model. The team had spent months, and about 10 million, training DBRX, a large language model similar in design to the one behind OpenAI's ChatGPT. But they wouldn't know how powerful their creation was until results came back from the final tests of its abilities. "We've surpassed everything," Jonathan Frankle, chief neural network architect at Databricks and leader of the team that built DBRX, eventually told the team, which responded with whoops, cheers, and applause emojis. Frankle usually steers clear of caffeine but was taking sips of iced latte after pulling an all-nighter to write up the results.

Jonathan Frankle is researching artificial intelligence -- not noshing pistachios -- but the same philosophy applies to his "lottery ticket hypothesis." It posits that, hidden within massive neural networks, leaner subnetworks can complete the same task more efficiently. The trick is finding those "lucky" subnetworks, dubbed winning lottery tickets. In a new paper, Frankle and colleagues discovered such subnetworks lurking within BERT, a state-of-the-art neural network approach to natural language processing (NLP). As a branch of artificial intelligence, NLP aims to decipher and analyze human language, with applications like predictive text generation or online chatbots.

BEGIN ARTICLE PREVIEW: You don’t need a sledgehammer to crack a nut. Jonathan Frankle is researching artificial intelligence — not noshing pistachios — but the same philosophy applies to his “lottery ticket hypothesis.” It posits that, hidden within massive neural networks, leaner subnetworks can complete the same task more efficiently. The trick is finding those “lucky” subnetworks, dubbed winning lottery tickets. In a new paper, Frankle and colleagues discovered such subnetworks lurking within BERT, a state-of-the-art neural network approach to natural language processing (NLP). As a branch of artificial intelligence, NLP aims to decipher and analyze human language, with applications like predictive te

You don't need a sledgehammer to crack a nut. Jonathan Frankle is researching artificial intelligence -- not noshing pistachios -- but the same philosophy applies to his "lottery ticket hypothesis." It posits that, hidden within massive neural networks, leaner subnetworks can complete the same task more efficiently. The trick is finding those "lucky" subnetworks, dubbed winning lottery tickets. In a new paper, Frankle and colleagues discovered such subnetworks lurking within BERT, a state-of-the-art neural network approach to natural language processing (NLP).

Jonathan Frankle and Michael Carbin, of Lottery Ticket fame, and Alex Renda, have made the perfect pruner, shrinking neural networks as much as you please, without sacrificing accuracy. And, the method is dead simple. When you add up the utility gained at the edge, these researchers are worth their weight in Californium. Paul Erdos, one of the greatest and by far the most prolific mathematician of the last century, would fall in love with those proofs and theorems which, by their utter simplicity, elegance, breadth of impact, and insightful technique, must have come from "The Book." By that, Erdos meant that scroll of truths God used to make the world.

Researchers have proposed a technique for shrinking deep learning models that they say is simpler and produces more accurate results than state-of-the-art methods. Massachusetts Institute of Technology (MIT) researchers have proposed a technique for compressing deep learning models, by retraining a smaller model whose weakest connections have been "pruned," at its faster, initial rate of learning. The technique's groundwork was partly laid by the AutoML for model compression (AMC) algorithm from MIT's Song Han, which automatically removes redundant neurons and connections, and retrains the model to reinstate its initial accuracy. MIT's Jonathan Frankle and Michael Carbin determined that the model could simply be rewound to its early training rate without tinkering with any parameters. Although greater shrinkage is accompanied by reduced model accuracy, in comparing their method to AMC or earlier work by Frankle on weight-rewinding techniques, Frankle and Carbin found that it performed better regardless of the amount of compression.

As more artificial intelligence applications move to smartphones, deep learning models are getting smaller to allow apps to run faster and save battery power. Now, MIT researchers have a new and better way to compress models. It's so simple that they unveiled it in a tweet last month: Train the model, prune its weakest connections, retrain the model at its fast, early training rate, and repeat, until the model is as tiny as you want. "That's it," says Alex Renda, a Ph.D. student at MIT. "The standard things people do to prune their models are crazy complicated." Renda discussed the technique when the International Conference of Learning Representations (ICLR) convened remotely this month.

As more artificial intelligence applications move to smartphones, deep learning models are getting smaller to allow apps to run faster and save battery power. Now, MIT researchers have a new and better way to compress models. It's so simple that they unveiled it in a tweet last month: Train the model, prune its weakest connections, retrain the model at its fast, early training rate, and repeat, until the model is as tiny as you want. "That's it," says Alex Renda, a PhD student at MIT. "The standard things people do to prune their models are crazy complicated." Renda discussed the technique when the International Conference of Learning Representations (ICLR) convened remotely this month.