Since the attention mechanism is rarely used within the GP literature, we provide a brief review of thetopicinthissection.

In this work, we address both of the above issues.

A Novel Reservoir Computing Framework for Chaotic Time Series Prediction Using Time Delay Embedding and Random Fourier Features S. K. Laha Advanced Design and Analysis Group CSIR - Central Mechanical Engineering Research Institute MG Avenue, Durgapur, West Bengal, PIN - 713209, India Abstract: Forecasting chaotic time series requires models that can capture the intrinsic geometry of the underlying attractor while remaining computationally efficient. We introduce a novel reservoir computing (RC) framework that integrates time - delay embedding with Random Fourier Feature (RFF) mappings to construct a dynamical reservoir without the need for traditional recurrent architectures. Unlike standard RC, which relies on high - dimensional recurrent connectivity, the proposed RFF - RC explicitly approximates non linear kernel transformations that uncover latent dynamical relations in the reconstructed phase space. This hybrid formulation offers two key advantages: (i) it provides a principled way to approximate complex nonlinear interactions among delayed coordina tes, thereby enriching the effective dynamical representation of the reservoir, and (ii) it reduces reliance on manual reservoir hyperparameters such as spectral radius and leaking rate. We evaluate the framework on canonical chaotic systems - the Mackey - Gla ss equation, the Lorenz system, and the Kuramoto - Sivashinsky equation. This novel formulation demonstrates that RFF - RC not only achieves superior prediction accuracy but also yields robust attractor reconstructions and long - horizon forecasts.

Spectral bias, the tendency of neural networks to learn low-frequency features first, is a well-known issue with many training algorithms for physics-informed neural networks (PINNs). To overcome this issue, we propose IFeF-PINN, an algorithm for iterative training of PINNs with Fourier-enhanced features. The key idea is to enrich the latent space using high-frequency components through Random Fourier Features. This creates a two-stage training problem: (i) estimate a basis in the feature space, and (ii) perform regression to determine the coefficients of the enhanced basis functions. For an underlying linear model, it is shown that the latter problem is convex, and we prove that the iterative training scheme converges. Furthermore, we empirically establish that Random Fourier Features enhance the expressive capacity of the network, enabling accurate approximation of high-frequency PDEs. Through extensive numerical evaluation on classical benchmark problems, the superior performance of our method over state-of-the-art algorithms is shown, and the improved approximation across the frequency domain is illustrated.

Estimators of HSIC typically assume i.i.d. A.3 Estimator of HSIC(Z, Z) Before discussing estimators of HSIC(Z,Z), note that it takes the following form: HSIC(Z,Z) = E null k (Z,Z Finally, note that even if null HSIC(Z,Z) is unbiased, its square root is not. B.1 InfoNCE connection To establish the connection with InfoNCE, define it in terms of expectations: L In the small variance regime, InfoNCE also bounds an HSIC-based loss. Both roots are real, as α 1 /4. Theorem B.1 works for any bounded kernel, because In Section 3.2, we make the assumption that the features are centered and argue that the assumption is valid for BYOL.

Since the attention mechanism is rarely used within the GP literature, we provide a brief review of the topic in this section. Below we follow the description of attention as given by V aswani et al. [8], including extensions to self-attention and multi-head self-attention. The dot-product attention mechanism [8] takes as input a set of queries, keys and values.

Recently, the information content (IC) of predictions from a Generative Infinite-Vocabulary Transformer (GIVT) has been used to model musical expectancy and surprisal in audio. We investigate the effectiveness of such modelling using IC calculated with autoregressive diffusion models (ADMs). We empirically show that IC estimates of models based on two different diffusion ordinary differential equations (ODEs) describe diverse data better, in terms of negative log-likelihood, than a GIVT. We evaluate diffusion model IC's effectiveness in capturing surprisal aspects by examining two tasks: (1) capturing monophonic pitch surprisal, and (2) detecting segment boundaries in multi-track audio. In both tasks, the diffusion models match or exceed the performance of a GIVT. We hypothesize that the surprisal estimated at different diffusion process noise levels corresponds to the surprisal of music and audio features present at different audio granularities. Testing our hypothesis, we find that, for appropriate noise levels, the studied musical surprisal tasks' results improve. Code is provided on github.com/SonyCSLParis/audioic.

It has been demonstrated that carefully designed reasoning paradigms, like Chain-of-Thought (CoT) and Tree-of-Thought (ToT), can enhance the reasoning capabilities of small language models by detailed thinking and extensive thought searching, unbounded branching factors in the searching space create prohibitive reasoning consumption. However these methods fall into the trap of local optimum reasoning, which means the model lacks a global perspective while solving problems. We propose a novel reasoning paradigm called Reason from Future (RFF), which generates reasoning paths by bidirectional reasoning that combines top-down planning with bottom-up reasoning accumulation. The essence of RFF lies in its reverse reasoning mechanism, which prioritizes core logical relationships and imposes goal-oriented constraints on intermediate steps, thereby reducing the searching space and mitigating error accumulation inherent in sequential forward reasoning. Empirical evaluations across diverse experiments demonstrate that RFF outperforms conventional paradigms with higher accuracy and less searching space to solve complex tasks.

Although Kolmogorov-Arnold based interpretable networks (KAN) have strong theoretical expressiveness, they face significant parameter explosion and high-frequency feature capture challenges in high-dimensional tasks. To address this issue, we propose the Kolmogorov-Arnold-Fourier Network (KAF), which effectively integrates trainable Random Fourier Features (RFF) and a novel hybrid GELU-Fourier activation mechanism to balance parameter efficiency and spectral representation capabilities. Our key technical contributions include: (1) merging KAN's dual-matrix structure through matrix association properties to substantially reduce parameters; (2) introducing learnable RFF initialization strategies to eliminate spectral distortion in high-dimensional approximation tasks; (3) implementing an adaptive hybrid activation function that progressively enhances frequency representation during the training process. Comprehensive experiments demonstrate the superiority of our KAF across various domains including vision, NLP, audio processing, and differential equation-solving tasks, effectively combining theoretical interpretability with practical utility and computational efficiency.