Chat-based language models are designed to be helpful, yet they should not comply with every user request. While most existing work primarily focuses on refusal of ``unsafe'' queries, we posit that the scope of noncompliance should be broadened. We introduce a comprehensive taxonomy of contextual noncompliance describing when and how models should comply with user requests.

Chat-based language models are designed to be helpful, yet they should not comply with every user request. While most existing work primarily focuses on refusal of "unsafe" queries, we posit that the scope of noncompliance should be broadened. We introduce a comprehensive taxonomy of contextual noncompliance describing when and how models should not comply with user requests. Our taxonomy spans a wide range of categories including incomplete, unsupported, indeterminate, and humanizing requests (in addition to unsafe requests). To test noncompliance capabilities of language models, we use this taxonomy to develop a new evaluation suite of 1000 noncompliance prompts. We find that most existing models show significantly high compliance rates in certain previously understudied categories with models like GPT-4 incorrectly complying with as many as 30% of requests. To address these gaps, we explore different training strategies using a synthetically-generated training set of requests and expected noncompliant responses. Our experiments demonstrate that while direct finetuning of instruction-tuned models can lead to both over-refusal and a decline in general capabilities, using parameter efficient methods like low rank adapters helps to strike a good balance between appropriate noncompliance and other capabilities.

We consider Dynamic Treatment Regimes (DTRs) with One Sided Noncompliance that arise in applications such as digital recommendations and adaptive medical trials. These are settings where decision makers encourage individuals to take treatments over time, but adapt encouragements based on previous encouragements, treatments, states, and outcomes. Importantly, individuals may not comply with encouragements based on unobserved confounders. For settings with binary treatments and encouragements, we provide nonparametric identification, estimation, and inference for Dynamic Local Average Treatment Effects (LATEs), which are expected values of multiple time period treatment contrasts for the respective complier subpopulations. Under standard assumptions in the Instrumental Variable and DTR literature, we show that one can identify Dynamic LATEs that correspond to treating at single time steps. Under an additional cross-period effect-compliance independence assumption, which is satisfied in Staggered Adoption settings and a generalization of them, which we define as Staggered Compliance settings, we identify Dynamic LATEs for treating in multiple time periods.

Causal mediation analysis (Pearl, 2001; VanderWeele, 2015; Imai et al., 2010a) is widely applied in experimental and observational studies to investigate the mechanism underlying a treatment-outcome relationship. Causal mediation methods have been developed under the potential outcomes framework with a primary objective to decompose the total treatment effect into an indirect effect that works through a specified mediator and a direct effect that works around the mediator. While alternative definitions exist, the natural indirect and direct effects have been considered as the most relevant for studying causal mechanisms (Nguyen et al., 2021). The natural indirect effect compares potential outcomes by switching the mediator from the value it would have taken under the control condition to the value it would have taken under the treated condition, while fixing the assignment to the treated condition. The natural direct effect compares potential outcomes by switching the assignment from the control condition to the treated condition, while fixing the mediator to the value it would have taken under the control condition. Parametric regressions (e.g., Valeri and VanderWeele, 2013; Cheng et al., 2021, 2023), semiparametric methods (e.g., Tchetgen Tchetgen and Shpitser, 2012), and nonparametric methods (e.g., Kim et al., 2017) have been proposed for estimating natural mediation effects, typically assuming that all study units perfectly comply with their treatment assignments. Experimental and observational studies are often subject to treatment noncompliance, where the actual treatment received for each unit may differ from the treatment assignment (Angrist et al., 1996). The intention-to-treat (ITT) effect (Lee et al., 1991) and the principal causal effect (PCE) (Frangakis and Rubin, 2002) represent two typical estimands to quantify the impact of intervention under noncompliance. To elaborate, the ITT estimand quantifies the'pragmatic effectiveness' of the treatment under real-world conditions, by measureing the effect of treatment assignment on the outcome among the study population regardless of the actual treatment receipt.

When it comes to keeping your workforce safe, providing proper personal protective equipment is just the start. Workplace safety programs of the future, however, will bring risk mitigation and incident prevention to the next level – including the ability to stop hazardous events from happening in the first place. With the help of artificial intelligence and workflow-integrated computer vision technology such as Benchmark Digital's new Benchmark ESG SuperVisionAI, the future is now. We've collaborated with world-class computer vision technology companies such as SparkCognition, Intenseye and 3MotionAI to harness the power of AI-analyzed live video streaming and photos to integrate the data they capture with the Benchmark solution platform to generate reports and alerts in real time for the ultimate safe work environment. The AI monitors and recognizes hazards as work happens, giving site leaders full visibility over key details such as PPE usage, potential hazards and day-to-day habits through a simple camera lens.

It is important to estimate the local average treatment effect (LATE) when compliance with a treatment assignment is incomplete. The previously proposed methods for LATE estimation required all relevant variables to be jointly observed in a single dataset; however, it is sometimes difficult or even impossible to collect such data in many real-world problems for technical or privacy reasons. We consider a novel problem setting in which LATE, as a function of covariates, is nonparametrically identified from the combination of separately observed datasets. For estimation, we show that the direct least squares method, which was originally developed for estimating the average treatment effect under complete compliance, is applicable to our setting. However, model selection and hyperparameter tuning for the direct least squares estimator can be unstable in practice since it is defined as a solution to the minimax problem. We then propose a weighted least squares estimator that enables simpler model selection by avoiding the minimax objective formulation. Unlike the inverse probability weighted (IPW) estimator, the proposed estimator directly uses the pre-estimated weight without inversion, avoiding the problems caused by the IPW methods. We demonstrate the effectiveness of our method through experiments using synthetic and real-world datasets.

The analysis, published this week in the proceedings of the Association of Computing Machinery Conference on Fairness, Accountability and Transparency(link is external), evaluates machine learning techniques designed to support a U.S. Environmental Protection Agency (EPA) initiative to reduce severe violations of the Clean Water Act. It reveals how two key elements of so-called algorithmic design influence which communities are targeted for compliance efforts and, consequently, who bears the burden of pollution violations. The analysis -- funded through the Stanford Woods Institute for the Environment's Realizing Environmental Innovation Program -- is timely given recent executive actions(link is external) calling for renewed focus on environmental justice. "Machine learning is being used to help manage an overwhelming number of things that federal agencies are tasked to do -- as a way to help increase efficiency," said study co-principal investigator Daniel Ho, the William Benjamin Scott and Luna M. Scott Professor of Law at Stanford Law School. "Yet what we also show is that simply designing a machine learning-based system can have an additional benefit."

Individual Treatment Effect (ITE) estimation is an extensively researched problem, with applications in various domains. We model the case where there exists heterogeneous non-compliance to a randomly assigned treatment, a typical situation in health (because of non-compliance to prescription) or digital advertising (because of competition and ad blockers for instance). The lower the compliance, the more the effect of treatment prescription, or individual prescription effect (IPE), signal fades away and becomes hard to estimate. We propose a new approach for the estimation of the IPE that takes advantage of observed compliance information to prevent signal fading. Using the Structural Causal Model framework and do-calculus, we define a general mediated causal effect setting and propose a corresponding estimator which consistently recovers the IPE with asymptotic variance guarantees. Finally, we conduct experiments on both synthetic and real-world datasets that highlight the benefit of the approach, which consistently improves state-of-the-art in low compliance settings