Collaborating Authors

MosaicML startup from Intel boss counting the cost of AI


A former head of artificial intelligence products at Intel has started a company to help companies cut overhead costs on AI systems. Naveen Rao, CEO and co-founder of MosaicML, previously led Nervana Systems, which was acquired by Intel for $350m. But like many Intel acquisitions, the marriage didn't pan out, and Intel killed the Nervana AI chip last year, after which Rao left the company. MosaicML's open source tools focus on implementing AI systems based on cost, training time, or speed-to-results. They do so by analyzing an AI problem relative to the neural net settings and hardware, which then paves an efficient path to generate optimal settings while reducing electric costs.

What Happens When Multipliers No Longer Define AI Accelerators?


Current custom AI hardware devices are built around super-efficient, high performance matrix multiplication. This category of accelerators includes the host of AI chip startups and defines what more mainstream accelerators like GPUs bring to the table. However, times might be changing as the role of matrix math tightens, making those devices weighted in the wrong direction, at least for areas in AI/ML that can trade in a little accuracy for speed. And while that may sound familiar (approximation) there are some new ideas on the horizon that blend the best of that old world of optimization and quantization and add a new twist--one that could dramatically reduce what's needed on a device. Instead of just approximating the way to efficient, fast AI, there are emerging algorithmic approaches that can take a known matrix and remove the multiply-add step altogether.

AI Weekly: AI model training costs on the rise, highlighting need for new solutions


This week, Microsoft and Nvidia announced that they trained what they claim is one of the largest and most capable AI language models to date: Megatron-Turing Natural Language Generation (MT-NLP). MT-NLP contains 530 billion parameters -- the parts of the model learned from historical data -- and achieves leading accuracy in a broad set of tasks, including reading comprehension and natural language inferences. But building it didn't come cheap. Experts peg the cost in the millions of dollars. Like other large AI systems, MT-NLP raises questions about the accessibility of cutting-edge research approaches in machine learning.

Automated Machine Learning with Monte-Carlo Tree Search (Extended Version) Machine Learning

The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset at hand. Mosaic, a Monte-Carlo tree search (MCTS) based approach, is presented to handle the AutoML hybrid structural and parametric expensive black-box optimization problem. Extensive empirical studies are conducted to independently assess and compare: i) the optimization processes based on Bayesian optimization or MCTS; ii) its warm-start initialization; iii) the ensembling of the solutions gathered along the search. Mosaic is assessed on the OpenML 100 benchmark and the Scikit-learn portfolio, with statistically significant gains over Auto-Sklearn, winner of former international AutoML challenges.

Using machine learning to explain extreme price moves


Understanding extreme asset price changes involves combining price history, news, events and social media data, much of which is only available in the form of unstructured text. By applying machine learning technologies to a real-time data pipeline, Refinitiv Labs has developed a prototype to help traders identify and respond to extreme price moves at pace. For more data-driven insights in your Inbox, subscribe to the Refinitiv Perspectives weekly newsletter. Data is abundant, not only in volume, but also in the number of sources it is derived from, the frequency at which it is updated, and the variety of formats it may take. Time spent sorting through that data, however, can keep businesses from generating actionable information at pace.