gretel
GRETEL: A Goal-driven Retrieval and Execution-based Trial Framework for LLM Tool Selection Enhancing
Wu, Zongze, Guo, Yani, Liang, Churong, Li, Runnan
Despite remarkable advances in Large Language Model capabilities, tool retrieval for agent-based systems remains fundamentally limited by reliance on semantic similarity, which fails to capture functional viability. Current methods often retrieve textually relevant but functionally inoperative tools due to parameter mismatches, authentication failures, and execution constraints--a phenomenon we term the semantic-functional gap. We introduce GRETEL, to address this gap through systematic empirical validation. GRETEL implements an agentic workflow that processes semantically retrieved candidates through sandboxed plan-execute-evaluate cycles, generating execution-grounded evidence to distinguish truly functional tools from merely descriptive matches. Our comprehensive evaluation on the ToolBench benchmark demonstrates substantial improvements across all metrics: Pass Rate (at 10) increases from 0.690 to 0.826, Recall (at 10) improves from 0.841 to 0.867, and NDCG (at 10) rises from 0.807 to 0.857.. These results establish that execution-based validation provides a more reliable foundation for tool selection than semantic similarity alone, enabling more robust agent performance in real-world applications.
- Asia > China > Hong Kong (0.05)
- Asia > China > Beijing > Beijing (0.05)
- North America > United States > California > San Francisco County > San Francisco (0.04)
- Asia > Thailand > Bangkok > Bangkok (0.04)
- Research Report (0.64)
- Workflow (0.50)
Do LLMs Align with My Task? Evaluating Text-to-SQL via Dataset Alignment
Rafiei, Davood, Heisler, Morgan Lindsay, Zhang, Weiwei, Pourreza, Mohammadreza, Zhang, Yong
Supervised Fine-Tuning (SFT) is an effective method for adapting Large Language Models (LLMs) on downstream tasks. However, variability in training data can hinder a model's ability to generalize across domains. This paper studies the problem of dataset alignment for Natural Language to SQL (NL2SQL or text to SQL), examining how well SFT training data matches the structural characteristics of target queries and how this alignment impacts model performance. We hypothesize that alignment can be accurately estimated by comparing the distributions of structural SQL features across the training set, target data, and the model's predictions prior to SFT. Through comprehensive experiments on three large cross-domain NL2SQL benchmarks and multiple model families, we show that structural alignment is a strong predictor of fine-tuning success. When alignment is high, SFT yields substantial gains in accuracy and SQL generation quality; when alignment is low, improvements are marginal or absent. These findings highlight the importance of alignment-aware data selection for effective fine-tuning and generalization in NL2SQL tasks.
- North America > Canada > British Columbia > Vancouver (0.14)
- Asia > Singapore (0.04)
- North America > United States > California > Alameda County (0.04)
- (6 more...)
Nvidia Bets Big on Synthetic Data
Nvidia has acquired synthetic data firm Gretel for nine figures, according to two people with direct knowledge of the deal. The acquisition price exceeds Gretel's most recent valuation of 320 million, the sources say, though the exact terms of the purchase remain unknown. Gretel and its team of approximately 80 employees will be folded into Nvidia, where its technology will be deployed as part of the chip giant's growing suite of cloud-based, generative AI services for developers. The acquisition comes as Nvidia has been rolling out synthetic data generation tools, so that developers can train their own AI models and fine-tune them for specific apps. In theory, synthetic data could create a near-infinite supply of AI training data and help solve the data scarcity problem that has been looming over the AI industry since ChatGPT went mainstream in 2022--although experts say using synthetic data in generative AI comes with its own risks.
Synthetic Data Is About To Transform Artificial Intelligence
Imagine if it were possible to produce infinite amounts of the world's most valuable resource, cheaply and quickly. What dramatic economic transformations and opportunities would result? This is a reality today. It is called synthetic data. Synthetic data is not a new idea, but it is now approaching a critical inflection point in terms of real-world impact. It is poised to upend the entire value chain and technology stack for artificial intelligence, with immense economic implications. Data is the lifeblood of modern artificial intelligence. Getting the right data is both the most important and the most challenging part of building powerful AI.
- Banking & Finance (1.00)
- Health & Medicine > Pharmaceuticals & Biotechnology (0.95)
- Information Technology > Security & Privacy (0.69)
GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization
Xie, Qianqian, Huang, Jimin, Saha, Tulika, Ananiadou, Sophia
Recently, neural topic models (NTMs) have been incorporated into pre-trained language models (PLMs), to capture the global semantic information for text summarization. However, in these methods, there remain limitations in the way they capture and integrate the global semantic information. In this paper, we propose a novel model, the graph contrastive topic enhanced language model (GRETEL), that incorporates the graph contrastive topic model with the pre-trained language model, to fully leverage both the global and local contextual semantics for long document extractive summarization. To better capture and incorporate the global semantic information into PLMs, the graph contrastive topic model integrates the hierarchical transformer encoder and the graph contrastive learning to fuse the semantic information from the global document context and the gold summary. To this end, GRETEL encourages the model to efficiently extract salient sentences that are topically related to the gold summary, rather than redundant sentences that cover sub-optimal topics. Experimental results on both general domain and biomedical datasets demonstrate that our proposed method outperforms SOTA methods.
Synthetic Data Is About To Transform Artificial Intelligence
These people do not exist. These faces were artificially generated using a form of deep learning ... [ ] known as generative adversarial networks (GANs). Synthetic data like this is becoming increasingly indistinguishable from real-world data. Imagine if it were possible to produce infinite amounts of the world's most valuable resource, cheaply and quickly. What dramatic economic transformations and opportunities would result? This is a reality today. It is called synthetic data. Synthetic data is not a new idea, but it is now approaching a critical inflection point in terms of real-world impact.
- Banking & Finance (1.00)
- Health & Medicine > Pharmaceuticals & Biotechnology (0.95)
- Information Technology > Security & Privacy (0.69)
GRETEL: A unified framework for Graph Counterfactual Explanation Evaluation
Prado-Romero, Mario Alfonso, Stilo, Giovanni
Machine Learning (ML) systems are a building part of the modern tools which impact our daily life in several application domains. Due to their black-box nature, those systems are hardly adopted in application domains (e.g. health, finance) where understanding the decision process is of paramount importance. Explanation methods were developed to explain how the ML model has taken a specific decision for a given case/instance. Graph Counterfactual Explanations (GCE) is one of the explanation techniques adopted in the Graph Learning domain. The existing works of Graph Counterfactual Explanations diverge mostly in the problem definition, application domain, test data, and evaluation metrics, and most existing works do not compare exhaustively against other counterfactual explanation techniques present in the literature. We present GRETEL, a unified framework to develop and test GCE methods in several settings. GRETEL is a highly extensible evaluation framework which promotes the Open Science and the evaluations reproducibility by providing a set of well-defined mechanisms to integrate and manage easily: both real and synthetic datasets, ML models, state-of-the-art explanation techniques, and evaluation measures. To present GRETEL, we show the experiments conducted to integrate and test several synthetic and real datasets with several existing explanation techniques and base ML models.
Career roadmap: Machine learning scientist
Like machine learning engineers, machine learning scientists are in high demand in today's job market. That's because organizations are eager to adopt machine learning-powered tools to enhance the value of their data and analytics and add automation to processes. Amy Steier, principal machine learning scientist at the developer tools provider, Gretel.ai. Demand for machine learning technologies is on the rise, according to market research. Potential applications include customer segmentation and investment prediction in the financial services sector; image analytics, drug discovery and personalized treatment in healthcare; and inventory planning and cross-channel marketing in retail.
- Information Technology > Security & Privacy (1.00)
- Education > Educational Setting (0.71)
- Banking & Finance (0.69)
- Media > Film (0.93)
- Leisure & Entertainment (0.68)