A former head of artificial intelligence products at Intel has started a company to help companies cut overhead costs on AI systems. Naveen Rao, CEO and co-founder of MosaicML, previously led Nervana Systems, which was acquired by Intel for $350m. But like many Intel acquisitions, the marriage didn't pan out, and Intel killed the Nervana AI chip last year, after which Rao left the company. MosaicML's open source tools focus on implementing AI systems based on cost, training time, or speed-to-results. They do so by analyzing an AI problem relative to the neural net settings and hardware, which then paves an efficient path to generate optimal settings while reducing electric costs.
Current custom AI hardware devices are built around super-efficient, high performance matrix multiplication. This category of accelerators includes the host of AI chip startups and defines what more mainstream accelerators like GPUs bring to the table. However, times might be changing as the role of matrix math tightens, making those devices weighted in the wrong direction, at least for areas in AI/ML that can trade in a little accuracy for speed. And while that may sound familiar (approximation) there are some new ideas on the horizon that blend the best of that old world of optimization and quantization and add a new twist--one that could dramatically reduce what's needed on a device. Instead of just approximating the way to efficient, fast AI, there are emerging algorithmic approaches that can take a known matrix and remove the multiply-add step altogether.
This week, Microsoft and Nvidia announced that they trained what they claim is one of the largest and most capable AI language models to date: Megatron-Turing Natural Language Generation (MT-NLP). MT-NLP contains 530 billion parameters -- the parts of the model learned from historical data -- and achieves leading accuracy in a broad set of tasks, including reading comprehension and natural language inferences. But building it didn't come cheap. Experts peg the cost in the millions of dollars. Like other large AI systems, MT-NLP raises questions about the accessibility of cutting-edge research approaches in machine learning.
The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset at hand. Mosaic, a Monte-Carlo tree search (MCTS) based approach, is presented to handle the AutoML hybrid structural and parametric expensive black-box optimization problem. Extensive empirical studies are conducted to independently assess and compare: i) the optimization processes based on Bayesian optimization or MCTS; ii) its warm-start initialization; iii) the ensembling of the solutions gathered along the search. Mosaic is assessed on the OpenML 100 benchmark and the Scikit-learn portfolio, with statistically significant gains over Auto-Sklearn, winner of former international AutoML challenges.
Understanding extreme asset price changes involves combining price history, news, events and social media data, much of which is only available in the form of unstructured text. By applying machine learning technologies to a real-time data pipeline, Refinitiv Labs has developed a prototype to help traders identify and respond to extreme price moves at pace. For more data-driven insights in your Inbox, subscribe to the Refinitiv Perspectives weekly newsletter. Data is abundant, not only in volume, but also in the number of sources it is derived from, the frequency at which it is updated, and the variety of formats it may take. Time spent sorting through that data, however, can keep businesses from generating actionable information at pace.