Goto

Collaborating Authors

 dcal


PLOT: Progressive Localization via Optimal Transport in Neural Causal Abstraction

arXiv.org Machine Learning

Causal abstraction offers a principled framework for mechanistic interpretability, aligning a high-level causal model with the low-level computation realized by a neural network through counterfactual intervention analysis. Existing methods such as distributed alignment search (DAS) learn expressive subspace interventions, but the relevant neural site is unknown a priori, so finding a handle requires a computationally burdensome search over candidate sites. We introduce PLOT (Progressive Localization via Optimal Transport), a transport-based framework that localizes causal variables from the output effect geometry of abstract and neural interventions. PLOT fits an optimal transport coupling between abstract variables and candidate neural sites, yielding a global soft correspondence that can be calibrated into intervention handles. In simple settings, a single coupling over individual neurons suffices. In larger models, PLOT is applied progressively, moving from coarse sites such as tokens, timesteps, or layers to finer supports such as coordinate groups or PCA spans, and optionally guiding DAS based on the localized signal. Across experiments of increasing complexity, transport-only PLOT handles are exceedingly fast and competitive on accuracy, while PLOT-guided DAS reaches DAS-level accuracy at a fraction of full DAS runtime, providing an efficient localization engine for causal abstraction research at scale.



31b3b31a1c2f8a370206f111127c0dbd-Supplemental.pdf

Neural Information Processing Systems

Note that we allow multiple estimated quantiles to be identical to eachother,to accommodate the possibility of point masses. Furthermore, we assume ˆq0(x) and ˆq1(x) are conservative upper and lower bounds for the support ofY | X = x, i.e., ˆq0(X) = b0 < Y < bm = ˆq1(X). We will discuss in the next section practical options for estimating ˆq(x). Now, we leverage any givenˆq(x) to compute estimatesˆπj(x) of the unknown bin probabilities πj(x) in (6), for allj {1,...,m}. Although there are multiple way of doing this, a principled solution is to convert the information contained inˆq into a piece-wise constant density estimate, and then integrate that density within each bin.


31b3b31a1c2f8a370206f111127c0dbd-Paper.pdf

Neural Information Processing Systems

This frameworkcanaccommodate almost anychoice of conformity scores, and in fact many different implementations have already been proposed to address ourproblem. However,itremains unclear howtoimplement aconcrete method fromthis broad family that can lead to the most informative possible prediction intervals.


Predictive Data Calibration for Linear Correlation Significance Testing

arXiv.org Artificial Intelligence

Inferring linear relationships lies at the heart of many empirical investigations. A measure of linear dependence should correctly evaluate the strength of the relationship as well as qualify whether it is meaningful for the population. Pearson's correlation coefficient (PCC), the \textit{de-facto} measure for bivariate relationships, is known to lack in both regards. The estimated strength $r$ maybe wrong due to limited sample size, and nonnormality of data. In the context of statistical significance testing, erroneous interpretation of a $p$-value as posterior probability leads to Type I errors -- a general issue with significance testing that extends to PCC. Such errors are exacerbated when testing multiple hypotheses simultaneously. To tackle these issues, we propose a machine-learning-based predictive data calibration method which essentially conditions the data samples on the expected linear relationship. Calculating PCC using calibrated data yields a calibrated $p$-value that can be interpreted as posterior probability together with a calibrated $r$ estimate, a desired outcome not provided by other methods. Furthermore, the ensuing independent interpretation of each test might eliminate the need for multiple testing correction. We provide empirical evidence favouring the proposed method using several simulations and application to real-world data.