Large language models (LLMs) have revolutionized machine learning due to their ability to capture complex interactions between input features. Popular post-hoc explanation methods like SHAP provide marginal feature attributions, while their extensions to interaction importances only scale to small input lengths ($\approx 20$). We propose Spectral Explainer (SPEX), a model-agnostic interaction attribution algorithm that efficiently scales to large input lengths ($\approx 1000)$. SPEX exploits underlying natural sparsity among interactions -- common in real-world data -- and applies a sparse Fourier transform using a channel decoding algorithm to efficiently identify important interactions. We perform experiments across three difficult long-context datasets that require LLMs to utilize interactions between inputs to complete the task. For large inputs, SPEX outperforms marginal attribution methods by up to 20% in terms of faithfully reconstructing LLM outputs. Further, SPEX successfully identifies key features and interactions that strongly influence model output. For one of our datasets, HotpotQA, SPEX provides interactions that align with human annotations. Finally, we use our model-agnostic approach to generate explanations to demonstrate abstract reasoning in closed-source LLMs (GPT-4o mini) and compositional reasoning in vision-language models.

Tim Maughan is a writer of fiction and nonfiction. His debut novel "Infinite Detail" (FSG, 2019) was The Guardian's science fiction and fantasy book of the year and was shortlisted for the Locus Award for best first novel. "Where'd you find this guy?" Dave Clutch asks me, pushing bangs out of anime-girl eyes with a flick of a cell-shaded hand. "Yeah, nice try lol," I reply. "Like I'm gonna tell you my trade secrets." I brace for a stylized sweat drop that never comes, thankfully. The aesthetic is so played out that my reaction to it borders on allergic, but at least it is an aesthetic. When I first found Dave he was just another youtuber, all skinny white boy grey skin and sagging eye bags lit by nothing but the slowly cycling RGB LEDs of his gaming rig, staring awkwardly into the camera as he read his scripts and flicked real bangs of hobbit hair out of his eyes. It was a look that screamed a desperate need for authenticity, and it was the first thing I had to beat out of him. Back then he was posting weekly videos about his patent dives into smart contact lens technology -- long, rambling monologues detailing what he'd unearthed about some obscure Chinese manufacturer and how they were going to "reinvent personal immersion" by making VR headsets and spex obsolete. What he didn't know was that the company was already in acquisition talks with Meta, and they'd been feeding him bullshit patents for fantasy tech in order to drive their market value up. Meta didn't give a fuck, the whole deal was pocket change for them, but the day traders whose algos had already decided his info was whack and the company was a good shorting opportunity were pissed and braying for blood. It didn't help that Dave had been stupid enough to blow his student loans on buying shares in them himself. It was not, as we used to say, a good look.

A new method for automated planning, progressive refinement of skeletal plans, has been developed for the problem of experiment design in the domain of molecular biology. The method resulted from a study of the problem-solving behavior of scientists which showed that design usually consisted of lookup of abstracted plans followe6 by hierarchical plan-step refinement. The skeletal plan method has been implemented through two generations of problem-solving systems: the second generation involving a synthesis with the metaplanning approach of Stefik.