Small Language Models for Application Interactions: A Case Study

Large Language Models (LLMs) are becoming pervasive in assisting humans with a wide variety of tasks, such as writing documents, presenting work, coding and health assistant. Generative LLMs are being rapidly integrated in user-facing software, for answering questions and increasing productivity through simple, language based interactions with technology. One of the key operating principles behind LLMs is exploiting their ability to generalize to unseen tasks by providing examples through the prompt itself - an approach commonly known as in-context learning. While LLMs are being designed to support larger prompt sizes, processing very large prompts might be expensive and incur non-negligible latencies. In this paper, we consider the alternative of using Small Language Models (SLMs), which are being developed nowadays and open-sourced by several companies.