Sample-and computationally-efficient distribution estimation is a fundamental tenet in statistics and machine learning.

Reasoning based on Large Language Models (LLMs) has garnered increasing attention due to outstanding performance of these models in mathematical and complex logical tasks. Beginning with the Chain-of-Thought (CoT) prompting technique, numerous reasoning methods have emerged that decompose problems into smaller, sequential steps (or thoughts). However, existing reasoning models focus on conventional problem-solving and do not necessarily generate creative solutions by ``creative reasoning''. In domains where the solution space is expansive and conventional solutions are suboptimal, such as drug discovery or business strategization, creative reasoning to discover innovative solutions is crucial. To address this gap, first we introduce a computational framework for creative reasoning inspired by established cognitive science principles. With this framework, we propose three core creative reasoning paradigms, namely, \textit{combinational}, \textit{exploratory}, and \textit{transformative} reasoning, where each offers specific directions for systematic exploration of the universe of thoughts to generate creative solutions. Next, to materialize this framework using LLMs, we introduce the \textit{Universe of Thoughts} (or \textit{UoT}, for short), a novel set of methods to implement the aforementioned three creative processes. Finally, we introduce three novel tasks that necessitate creative problem-solving, along with an evaluation benchmark to assess creativity from three orthogonal perspectives: feasibility as constraint, and utility and novelty as metrics. With a comparative analysis against the state-of-the-art (SOTA) reasoning techniques as well as representative commercial models with reasoning capability, we show that UoT demonstrates superior performance in creative reasoning.

We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse.

Deep vision models perform input-output computations that are hard to interpret. Concept-based explanation methods (CBEMs) increase interpretability by re-expressing parts of the model with human-understandable semantic units, or concepts. Checking if the derived explanations are faithful -- that is, they represent the model's internal computation -- requires a surrogate that combines concepts to compute the output. Simplifications made for interpretability inevitably reduce faithfulness, resulting in a tradeoff between the two. State-of-the-art unsupervised CBEMs (U-CBEMs) have reported increasingly interpretable concepts, while also being more faithful to the model. However, we observe that the reported improvement in faithfulness artificially results from either (1) using overly complex surrogates, which introduces an unmeasured cost to the explanation's interpretability, or (2) relying on deletion-based approaches that, as we demonstrate, do not properly measure faithfulness. We propose Surrogate Faithfulness (SURF), which (1) replaces prior complex surrogates with a simple, linear surrogate that measures faithfulness without changing the explanation's interpretability and (2) introduces well-motivated metrics that assess loss across all output classes, not just the predicted class. We validate SURF with a measure-over-measure study by proposing a simple sanity check -- explanations with random concepts should be less faithful -- which prior surrogates fail. SURF enables the first reliable faithfulness benchmark of U-CBEMs, revealing that many visually compelling U-CBEMs are not faithful. Code to be released.

We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse.

Ideally, we would prefer an estimator that learns any distribution.

The conic formulation detailed in Section A.3 is obtained by performing the optimal transport on ( x, 0) Note that Liero et al. [2015] do not mention that this The proofs are detailed in Liero et al. [2015]. We first start with the existence of minimizers stated in Proposition 1. Thus it suffices to have relative compactness of the set of minimizers. There exists a Borel measurable bijection between the measures' supports It is the same proof as in the main body. We present in this section the proofs of the properties mentioned in Section 2. We refer to Section 2 In this section we frequently use the notion of marginal for neasures. We present in this section concepts and properties which are necessary for the proof of Theorem 1.

Ideally, we would prefer an estimator that learns any distribution.

The conic formulation detailed in Section A.3 is obtained by performing the optimal transport on ( x, 0) Note that Liero et al. [2015] do not mention that this The proofs are detailed in Liero et al. [2015]. We first start with the existence of minimizers stated in Proposition 1. Thus it suffices to have relative compactness of the set of minimizers. There exists a Borel measurable bijection between the measures' supports It is the same proof as in the main body. We present in this section the proofs of the properties mentioned in Section 2. We refer to Section 2 In this section we frequently use the notion of marginal for neasures. We present in this section concepts and properties which are necessary for the proof of Theorem 1.

This is something that should be explained. The "COMP" procedure looks complicated at a first sight, but is quite common actually.