The recent success of large language models (LLMs) has exposed the vulnerability of public data as these models are trained on data scraped at scale from public forums and news articles [Touvron et al., 2023] without consent, and the collection of this data remains largely unregulated. As a result, governments worldwide have passed several regulatory frameworks, such as the GDPR [Voigt and Von dem Bussche, 2017] in the EU, the Personal Information Protection and Electronic Documents Act in Canada [PIPEDA], the Data Protection Act in the UK [DPA], the Personal Data Protection Commission (PDPC) [Commission et al., 2022] in Singapore, and the EU AI Act [Neuwirth, 2022], to safeguard algorithmic decisions and data usage practices. The aforementioned legislative frameworks emphasize individuals' rights over how their data is used, even in public contexts. These laws are not limited to private or sensitive data but also encompass the ethical use of publicly accessible information, especially in contexts where such data is used for profiling, decision-making, or large-scale commercial gains. Despite the regulatory efforts, state-of-the-art LLMs are increasingly used in real-world applications to exploit personal data and predict political affiliations [Rozado, 2024, Hernandes, 2024], societal biases [Liang et al., 2021, Dong et al., 2024], and sensitive information of individuals [Wan et al., 2023b, Salewski et al., 2024, Suman et al., 2021], highlighting significant gaps between research and regulatory frameworks. In this work, we aim to make the first attempt to operationalize one principle of "right to protect data" into algorithmic implementation in practice, i.e., people having control over their online data, and propose R

Existing debiasing techniques are typically training-based or require access to the model's internals and output distributions, so they are inaccessible to end-users looking to adapt LLM outputs for their particular needs. In this study, we examine whether structured prompting techniques can offer opportunities for fair text generation. We evaluate a comprehensive end-user-focused iterative framework of debiasing that applies System 2 thinking processes for prompts to induce logical, reflective, and critical text generation, with single, multi-step, instruction, and role-based variants. By systematically evaluating many LLMs across many datasets and different prompting strategies, we show that the more complex System 2-based Implicative Prompts significantly improve over other techniques demonstrating lower mean bias in the outputs with competitive performance on the downstream tasks. Our work offers research directions for the design and the potential of end-user-focused evaluative frameworks for LLM use.

Fairness in Language Models (LMs) remains a longstanding challenge, given the inherent biases in training data that can be perpetuated by models and affect the downstream tasks. Recent methods employ expensive retraining or attempt debiasing during inference by constraining model outputs to contrast from a reference set of biased templates or exemplars. Regardless, they dont address the primary goal of fairness to maintain equitability across different demographic groups. In this work, we posit that inferencing LMs to generate unbiased output for one demographic under a context ensues from being aware of outputs for other demographics under the same context. To this end, we propose Counterfactually Aware Fair InferencE (CAFIE), a framework that dynamically compares the model understanding of diverse demographics to generate more equitable sentences. We conduct an extensive empirical evaluation using base LMs of varying sizes and across three diverse datasets and found that CAFIE outperforms strong baselines. CAFIE produces fairer text and strikes the best balance between fairness and language modeling capability