Perceive, the AI chip startup spun out of Xperi, has released a second chip with hardware support for transformers, including large language models (LLMs) at the edge. The company demonstrated sentence completion via RoBERTa, a transformer network with 110 million parameters, on its Ergo 2 chip at CES 2023. Ergo 2 comes in the same 7mm x 7mm package as the original Ergo, but offers roughly 4 the performance. This performance increase translates to edge inference of transformers with more than 100 million parameters, video processing at higher frame rates or inference of multiple large neural networks at once. For example, the YoloV5-S inference can run at up to 115 inferences per second on Ergo 2; YoloV5-S inference at 30 images per second requires just 75 mW.

Embedded vision technologies are giving machines the power of sight, but today's systems still fall short of understanding all the nuances of an image. An approach used for natural language processing could address that. Attention-based neural networks, particularly transformer networks, have revolutionized natural language processing (NLP), giving machines a better understanding of language than ever before. This technique, which is designed to mimic cognitive processes by giving an artificial neural network an idea of history or context, has produced much more sophisticated AI agents than older approaches that also employ memory, such as long short-term memory (LSTM) and recurrent neural networks (RNNs). NLP now has a deeper level of understanding of the questions or prompts it is fed and can create long pieces of text in response that are often indistinguishable from what a human might write.

Machine learning (ML) is about making predictions about new data based on old data. The quality of any machine-learning algorithm is ultimately determined by the quality of those predictions. However, there is no one universal way to measure that quality across all ML applications, and that has broad implications for the value and usefulness of machine learning. "Every industry, every domain, every application has different care-abouts," said Nick Ni, director of product marketing, AI and software at Xilinx. "And you have to measure that care-about." Classification is the most familiar application, and "accuracy" is the measure used for it. But even so, there remain disagreements about exactly how accuracy should be measured or what it should mean. With other applications, it's much less clear how to measure the quality of results.