Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight. In an era of rapidly increasing computational resources for pre-training and a near stagnation in the generation of new real-world data in many applications, PFNs are poised to play a more important role across a wide range of applications. They enable the efficient allocation of pre-training compute to low-data scenarios. Originally applied to small Bayesian modeling tasks, the field of PFNs has significantly expanded to address more complex domains and larger datasets. This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference, leveraging amortized learning to tackle data-scarce problems. We thus believe they are a fruitful area of research. In this position paper, we explore their potential and directions to address their current limitations.

Summary: Within the manuscript, the authors extend the continuous time Bayesian Networks by incorporating a mixture prior over the conditional intensity matrices, thereby allowing for a larger class compared to a gamma prior usually employed over these. My main concerns are with clarity / quality as the manuscript is quite densely written with quite some material has either been omitted or shifted to the appendix. For a non-expert in continuous time bayesian networks, it is quite hard to read. Additionally, there are quite a few minor mistakes (see below) that make understanding of the manuscript harder. As it stands, Originality: The authors combine variational inference method from Linzner et al [11], with the new prior over the dependency structure (mixture).

Summary and Contributions: This paper introduces a version of Zap Q-learning that can be applied to arbitrary approximation architectures for Q-functions. Convergence analysis is undertaken, and a version of the algorithm with MLP function approximators is applied to several classical control tasks. POST-REBUTTAL ------------------------ I thank the authors for their response. I appreciate the comments around reorganisation of material, and clarification of some of the technical points I raised. There are two main concerns that I have with the paper that prevent me from strongly recommending acceptance, described below.

The reviewers are generally supportive of the paper. They have provided some very useful feedback, and I highly encourage the authors to incorporate that feedback. Primarily, it would be ideal to complete the paper reorganization as discussed, explain the limitations in the assumption on boundedness of the iterates, provide a toy example where the boundness assumption is not on its own enough to prevent divergence of Q-learning (i.e, even under that assumption, Q-learning diverges but Zap-Q does not) and finally to sweep over the parameters in the empirical comparison (even if that means the outcome is less positive for Zap-Q).

Additional Feedback: I think the overall idea is somewhat intriguing, and agree that the distinction between hard prediction and model-assigned soft probabilities is typically glossed over in the literature. But sections 3 and 4 seems somewhat disconnected to me, since this differentiability issues does not come across strongly in the experiments (instead the instability of baselines due to adversarial training is emphasized). I also think that laying out the extending to multi-class classification would help the paper a lot (to save space experimental details could be moved to the appendix) About the experiments, there are several improvements to be made. I have a concern about whether the baselines were substantially tuned, since there is no discussion of how hyperparameters such as learning rates were selected for the baselines. I also didn't understand the choice of a narrow MLP architecture (14 hidden units per layer) for the neural net methods.

The paper proposes a simple but rather practical approach to estimate statistical fairness notions without relying on a proxy, in contrast to several prior work. The proposed approach relies on Kernel Density Estimation (KDE), which allows to compute the gradient of the fairness notion with respect to the model parameters in close form, easing the learning procedure of a fair classifier. As a result, he proposed approach leads to a better fairness accuracy trade-off than competing methods in several datasets. In particular, the experiments show that the proposed approach outperforms prior work relying on fairness proxies, and leads more stable results that approaches that rely on adversarial training top trade-off fairness and accuracy. In fact, the empirical results are comparable to the ones provided by Agarwal et al. (2018), whose solution provide theoretical guarantees but comes at a high computational cost. Although there exists extensive literature on solving the fair classification problem, the empirical results show the efficacy of KDE in this context.

Dropout regularization methods prune a neural network's pre-determined backbone structure to avoid overfitting. However, a deep model still tends to be poorly calibrated with high confidence on incorrect predictions. We propose a unified Bayesian model selection method to jointly infer the most plausible network depth warranted by data, and perform dropout regularization simultaneously. In particular, to infer network depth we define a beta process over the number of hidden layers which allows it to go to infinity. Layer-wise activation probabilities induced by the beta process modulate neuron activation via binary vectors of a conjugate Bernoulli process. Experiments across domains show that by adapting network depth and dropout regularization to data, our method achieves superior performance comparing to state-of-the-art methods with well-calibrated uncertainty estimates. In continual learning, our method enables neural networks to dynamically evolve their depths to accommodate incrementally available data beyond their initial structures, and alleviate catastrophic forgetting.

Electrifying space and water heating is a critical priority for the energy transition. The necessary widespread adoption of heat pumps will have significant impacts on the power grid. Studies report heating electrification may increase winter peak electricity demand by up to 70%, with some colder regions experiencing a more than fourfold increase in peak demand. The process of upgrading the grid has a critical spatial dimension, as heating demand, electricity demand, and the capacity of existing grid infrastructure vary significantly across regions. Grid planning also involves a critical temporal dimension: short-term weather patterns and long-term climate change introduce complexities and uncertainties that can be di fficult to quantify. However, most existing demand forecasts are provided and validated only at aggregated spatial scales, lack temporal detail, and provide single-valued predictions. Without accurate, probabilistic, and spatially and temporally resolved demand forecasts, planners risk misallocating scarce resources. We present a novel framework for high-resolution forecasting of residential heating and electricity demand using probabilistic deep learning models. We focus specifically on providing hourly building-level electricity and heating demand forecasts for the residential sector. Leveraging multimodal building-level information - including data on building footprint areas, heights, nearby building density, nearby building size, land use patterns, and high-resolution weather data - and probabilistic modeling, our methods provide granular insights into demand heterogeneity. V alida-tion at the building level underscores a step change improvement in performance relative to NREL's ResStock model, which has emerged as a research community standard for residential heating and electricity demand characterization. In building-level heating and electricity estimation backtests, our probabilistic models respectively achieve RMSE scores 18.3% and 35.1% lower than those based on ResStock. Introduction Electrifying space and water heating is a critical priority for the energy transition [1, 2]. Residential and commercial buildings make up 13% of all U.S. emissions [3], with fossil-fueled space heating representing the single greatest constituent of this share [4]. The necessary widespread adoption of heat pumps will have significant impacts on the power grid.

When data are noisy, a statistical learner's goal is to resolve epistemic uncertainty about the data it will encounter at test-time, i.e., to identify the distribution of test (target) data. Many real-world learning settings introduce sources of epistemic uncertainty that can not be resolved on the basis of training (source) data alone: The source data may arise from multiple tasks (multitask learning), the target data may differ systematically from the source data tasks (distribution shift), and/or the learner may not arrive at an accurate characterization of the source data (imperfect learning). We introduce a principled definition of epistemic error, and provide a generic, decompositional epistemic error bound. Our error bound is the first to (i) consider epistemic error specifically, (ii) accommodate all the sources of epistemic uncertainty above, and (iii) separately attribute the error to each of multiple aspects of the learning procedure and environment. As corollaries of the generic result, we provide (i) epistemic error bounds specialized to the settings of Bayesian transfer learning and distribution shift within $ε$-neighborhoods, and (ii) a set of corresponding generalization bounds. Finally, we provide a novel definition of negative transfer, and validate its insights in a synthetic experimental setting.

Density ratio estimation is fundamental to tasks involving f-divergences, yet existing methods often fail under significantly different distributions or inadequately overlapping supports -- the density-chasm and the support-chasm problems. Additionally, prior approaches yield divergent time scores near boundaries, leading to instability. We design $\textbf{D}^3\textbf{RE}$, a unified framework for robust, stable and efficient density ratio estimation. We propose the dequantified diffusion bridge interpolant (DDBI), which expands support coverage and stabilizes time scores via diffusion bridges and Gaussian dequantization. Building on DDBI, the proposed dequantified Schr{ö}dinger bridge interpolant (DSBI) incorporates optimal transport to solve the Schr{ö}dinger bridge problem, enhancing accuracy and efficiency. Our method offers uniform approximation and bounded time scores in theory, and outperforms baselines empirically in mutual information and density estimation tasks.