Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendations to generative tasks. Although scaling laws indicate that larger models generally yield better generalization and performance, their substantial computational requirements often render them impractical for many real-world scenarios at scale. In this paper, we present methods and insights for training small language models (SLMs) that deliver high performance and efficiency in deployment. We focus on two key techniques: (1) knowledge distillation and (2) model compression via quantization and pruning. These approaches enable SLMs to retain much of the quality of their larger counterparts while significantly reducing training, serving costs, and latency. We detail the impact of these techniques on a variety of use cases at a large professional social network platform and share deployment lessons - including hardware optimization strategies that enhance speed and throughput for both predictive and reasoning-based applications.

Over its lifetime, the average car is responsible for emitting about 126,000 pounds of the greenhouse gas carbon dioxide (CO2). Compare those emissions with the carbon footprint left behind by artificial intelligence (AI) technology. In 2019, training top-of-the-line artificial intelligence was responsible for more than 625,000 pounds of CO2 emissions. AI energy requirements have only gotten bigger since. To reduce AI's energy footprint, Shantanu Chakrabartty, the Clifford W. Murphy Professor at the McKelvey School of Engineering at Washington University in St. Louis, has reported a prototype of a new kind of computer memory.

A deep-down analysis reveals that the company indeed has some key … the company's highly efficient machine learning process in the medium term.

University of Tokyo create a new integrated three-dimensional circuit architecture for artificial intelligence applications with spiraling stacks of memory modules. Researchers at the University of Tokyo Institute of Industrial Science in Japan stacked resistive random-access memory modules for artificial intelligence (AI) applications in a novel three-dimensional spiral. The modules feature oxide semiconductor access transistors, which boost the efficiency of the machine learning training process. The team further enhanced energy efficiency via a system of binarized neural networks, which restricts the parameters to be either 1 or -1, rather than any number, to compress the volume of data to be stored. In having the device interpret a database of handwritten digits, the researchers learned that increasing the size of each circuit layer could improve algorithmic accuracy to approximately 90%.