Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection

Juttu, Noshitha Padma Pratyusha, Singireddy, Sahithi, Gona, Sravani, Timilsina, Sujal

arXiv.org Artificial Intelligence 

T erms of Service (T oS) agreements often contain clauses that are difficult to interpret and potentially unfair to users. Manual identification of such clauses is infeasible at scale, motivating the need for automated, accurate, and efficient detection methods. This study presents a comprehensive evaluation of clause-level unfairness detection using a diverse range of large language model (LLM) strategies, including full fine-tuning, parameter-efficient tuning, and zero-shot prompting. Experiments are conducted with full fine-tuning on BERT and DistilBERT, 4-bit quantized Low-Rank Adaptation (LoRA) applied to models such as TinyLlama and LLaMA, and to the legal domain-specific SaulLM, and evaluate zero-shot prompting using high-performing API-accessible models like GPT-4o and O3-mini. Evaluations are performed on the Claudette-T oS dataset from Hugging Face and further validated on the Multilingual Scraper of Privacy Policies and T erms of Service corpus, which comprises large-scale T oS documents collected from the web. Full fine-tuning delivers the strongest overall performance, parameter-efficient models offer a favorable accuracy-efficiency trade-off, and zero-shot prompting enables fast deployment with high recall. These results offer practical insights into building scalable and cost-effective unfairness detection systems for legal-tech applications.

Duplicate Docs Excel Report

Title
None found

Similar Docs  Excel Report  more

TitleSimilaritySource
None found