Metaheuristics and Large Language Models Join Forces: Towards an Integrated Optimization Approach

The advent of Large Language Models (LLMs) has altered the Natural Language Processing (NLP) landscape, empowering professionals across diverse disciplines with their remarkable ability to generate human-like text. Models like OpenAI's GPT [44], Meta's Llama [45], and Anthropic's Claude 3 [4] have become indispensable collaborators in many peoples' daily lives; giving rise to innovative products such as ChatGPT for general use, GitHub Copilot for code generation, DALL-E 2 for image creation, and a multitude of voice generators, including OpenAI's text-to-speech API and ElevenLabs's Generative Voice AI. Currently, LLMs are being experimentally applied across various fields, yielding mixed results [3]. While some applications seem questionable, others exhibit spectacular outcomes. One of the most contentious applications is using LLMs for tasks necessitating mathematical reasoning. Given LLMs' inherently probabilistic nature, this application was once deemed implausible. However, recent findings suggest a shift in perspective, particularly with LLMs boasting vast parameter counts [1]. As LLMs continue to scale, new capabilities emerge [48]. Crucially, these opportunities are contingent upon the thoughtful design of prompts, which helps mitigate the risk of LLMs providing irrelevant or inaccurate responses [47]. 1