Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollers

Despite many recent successes with previous-generation Deep The latest evolutions in mainstream Artificial Intelligence (AI) Neural Networks (DNNs), the emergence of the tinyML paradigm have been driven by Transformers, which have taken over from for EFMs faces the dual challenge of reducing FMs to a manageable Recurrent Neural Networks (RNNs) and Convolutional Neural size and enabling their deployment on tiny devices. Networks (CNNs) as the leading edge models for language A first concrete step in this direction is the recent introduction of processing and multi-modal applications [1], [2]. The success of Small Language Models (SLMs): FMs with tens to a few hundred Transformers can be primarily attributed to the emergence of the million, rather than several billion parameters [8], [9]. While Foundation Model (FM) paradigm: large Transformer models most currently available FMs are focused on processing natural extensively pre-trained on datasets spanning trillions of tokens and language at a proof-of-concept scale, the effort towards embedded then fine-tuned with a much lower volume of labeled data to solve multi-modal sensor inputs with small-scale, application-specific domain-specific problems. Following the success of FMs in Natural FMs offers a highly promising path for the development of this Language Processing (NLP) [1], [3], an increasing number of fields novel class of models.