Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey

In the era of big data, machine learning (ML) workflows have become essential across various sectors for processing and analyzing large-scale data Xin et al. [2021], Nikitin et al. [2022]. To support the development and sharing of ML workflows, numerous repositories have been established, showcasing diverse paradigms for data analysis. For instance, KNIME offers a repository with over 25,000 workflows and 2,200 components Ordenes and Silipo [2021], providing a comprehensive collection of rigorously tested, practical models complete with detailed specifications. However, despite the availability of these resources, manually constructing and optimizing workflows to meet complex task requirements remains a knowledge-intensive and time-consuming challenge for most people. The advent of Large Language Models (LLMs) has recently revolutionized artificial intelligence (AI) and ML, delivering advanced capabilities in natural language understanding and generation Hollmann et al. [2024], Wang et al. [2024a]. Models such as OpenAI's GPT-4 Achiam et al. [2023] and Meta AI's LLaMA-3 Touvron et al. [2023] have demonstrated exceptional performance across a wide range of natural language processing (NLP) tasks, thanks to their extensive training on large-scale text datasets. Additionally, multimodal LLMs Hu et al. [2024], Tai et al. [2024], Luo et al. [2024], which incorporate various data types like audio and images, allow for richer interactions by processing and generating non-textual information. Their impressive capabilities have led to widespread adoption across multiple domains Gu et al. [2023], Klievtsova et al. [2023], Zhang et al. [2023a].