Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning

We use decision trees to convey this reasoning information, as they can be easily represented in natural language, effectively providing knowledge from prior experiments ( i.e., the impact of the generated features on performance) to