We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with reverse dynamic method (known in literature as "adjoint method") to train neural ODEs on classification, density estimation and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method what was confirmed and validated by extensive numerical experiments for several standard benchmarks.

We want to thank the reviewers for their thorough comments and constructive suggestions! We respond to the reviewers' comments below and update the final version of the Datasets considered for the density estimation task are only synthetic. Shampine (1985) yield that we get 4th-order approximations at pre-defined grid points. ANODE exploits the checkpointing technique in the backward pass (see Figure 1d). Therefore, we did not compare with ANODE since the out of memory error was raised during training.

Weaknesses: I take issues with two aspects of this submission that lead me to recommend rejection at this point. The submission points out that the evaluations of z(t) at the Chebyshev grid points can be obtained without additional cost, e.g., at line 108. While this is true in some sense in general, there are many numerical theory aspects to this claim that are ignored here, both in the text as well as in the code. Runge-Kutta methods only guarantee high-order approximations at their own grid points. If high-order approximations are sought at pre-defined grid points, there are two solutions: a) the solvers are forced to include the pre-defined grid points as part of the otherwise adaptive mesh or b) a particular choice has to be made to find a smooth-interpolant Runge-Kutta formula.

The four reviewers, all of whom are domain experts, agree that this is a good paper that delivers a delicate but useful methodological contribution to the growing area of NODEs. It should thus be accepted However, the reviewers have also raised several suggestions and requests for improvements. Please make sure to address them as much as possible to ensure this paper reaches its audience.

This paper discussed how to enhance the existing methods in which designed prior could over regularize the posteriori, so it will try to find a way to learn a complex prior which can learn the latent pattern of data manifold more efficiently. To learn such prior, paper adopted and modified one dual optimization technique and introduced an efficient algorithm on how to update the hierarchical prior and posteriori parameters. The combination of complex priori with the introduced algorithm have learned a posterior which has more informative latent representation and avoids posteriori collapse. In addition, paper introduced a graph search method to interpolate the states and showed how effective algorithm can discover a meaningful posteriori over the experiment section. So we can summarize the contribution of this paper as following - Introduce a hierarchical prior which can avoid over regularization of the posterior while learning latent variables manifold - Adopting and expanding an optimization technique and an algorithm to learn hierarchical prior and hierarchical posterior parameters.

Data augmentation is a crucial step in the development of robust supervised learning models, especially when dealing with limited datasets. This study explores interpolation techniques for the augmentation of geo-referenced data, with the aim of predicting the presence of Commelina benghalensis L. in sugarcane plots in La R{\'e}union. Given the spatial nature of the data and the high cost of data collection, we evaluated two interpolation approaches: Gaussian processes (GPs) with different kernels and kriging with various variograms. The objectives of this work are threefold: (i) to identify which interpolation methods offer the best predictive performance for various regression algorithms, (ii) to analyze the evolution of performance as a function of the number of observations added, and (iii) to assess the spatial consistency of augmented datasets. The results show that GP-based methods, in particular with combined kernels (GP-COMB), significantly improve the performance of regression algorithms while requiring less additional data. Although kriging shows slightly lower performance, it is distinguished by a more homogeneous spatial coverage, a potential advantage in certain contexts.

We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with reverse dynamic method (known in literature as "adjoint method") to train neural ODEs on classification, density estimation and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method what was confirmed and validated by extensive numerical experiments for several standard benchmarks.

Effective verification and validation techniques for modern scientific machine learning workflows are challenging to devise. Statistical methods are abundant and easily deployed, but often rely on speculative assumptions about the data and methods involved. Error bounds for classical interpolation techniques can provide mathematically rigorous estimates of accuracy, but often are difficult or impractical to determine computationally. In this work, we present a best-of-both-worlds approach to verifiable scientific machine learning by demonstrating that (1) multiple standard interpolation techniques have informative error bounds that can be computed or estimated efficiently; (2) comparative performance among distinct interpolants can aid in validation goals; (3) deploying interpolation methods on latent spaces generated by deep learning techniques enables some interpretability for black-box models. We present a detailed case study of our approach for predicting lift-drag ratios from airfoil images. Code developed for this work is available in a public Github repository.

This paper focuses on the hypothesis of optimizing time series predictions using fractal interpolation techniques. In general, the accuracy of machine learning model predictions is closely related to the quality and quantitative aspects of the data used, following the principle of \textit{garbage-in, garbage-out}. In order to quantitatively and qualitatively augment datasets, one of the most prevalent concerns of data scientists is to generate synthetic data, which should follow as closely as possible the actual pattern of the original data. This study proposes three different data augmentation strategies based on fractal interpolation, namely the \textit{Closest Hurst Strategy}, \textit{Closest Values Strategy} and \textit{Formula Strategy}. To validate the strategies, we used four public datasets from the literature, as well as a private dataset obtained from meteorological records in the city of Brasov, Romania. The prediction results obtained with the LSTM model using the presented interpolation strategies showed a significant accuracy improvement compared to the raw datasets, thus providing a possible answer to practical problems in the field of remote sensing and sensor sensitivity. Moreover, our methodologies answer some optimization-related open questions for the fractal interpolation step using \textit{Optuna} framework.

Statistical mechanics of spin glasses is one of the main strands toward a comprehension of information processing by neural networks and learning machines. Tackling this approach, at the fairly standard replica symmetric level of description, recently Hebbian attractor networks with multi-node interactions (often called Dense Associative Memories) have been shown to outperform their classical pairwise counterparts in a number of tasks, from their robustness against adversarial attacks and their capability to work with prohibitively weak signals to their supra-linear storage capacities. Focusing on mathematical techniques more than computational aspects, in this paper we relax the replica symmetric assumption and we derive the one-step broken-replica-symmetry picture of supervised and unsupervised learning protocols for these Dense Associative Memories: a phase diagram in the space of the control parameters is achieved, independently, both via the Parisi's hierarchy within then replica trick as well as via the Guerra's telescope within the broken-replica interpolation. Further, an explicit analytical investigation is provided to deepen both the big-data and ground state limits of these networks as well as a proof that replica symmetry breaking does not alter the thresholds for learning and slightly increases the maximal storage capacity. Finally the De Almeida and Thouless line, depicting the onset of instability of a replica symmetric description, is also analytically derived highlighting how, crossed this boundary, the broken replica description should be preferred.