We investigate how well a physics-based simulator can replicate a real wheel loader performing bucket filling in a pile of soil. The comparison is made using field test time series of the vehicle motion and actuation forces, loaded mass, and total work. The vehicle was modeled as a rigid multibody system with frictional contacts, driveline, and linear actuators. For the soil, we tested discrete element models of different resolutions, with and without multiscale acceleration. The spatio-temporal resolution ranged between 50-400 mm and 2-500 ms, and the computational speed was between 1/10,000 to 5 times faster than real-time. The simulation-to-reality gap was found to be around 10% and exhibited a weak dependence on the level of fidelity, e.g., compatible with real-time simulation. Furthermore, the sensitivity of an optimized force feedback controller under transfer between different simulation domains was investigated. The domain bias was observed to cause a performance reduction of 5% despite the domain gap being about 15%.

The mode shape function is difficult to determine in modeling manipulators with flexible links using the assumed mode method. In this paper, for a planar 3-RRR parallel manipulator with flexible actuation links, we provide a data-driven method to identify the mode shape of the flexible links and propose a model-based controller for the vibration suppression. By deriving the inverse kinematics of the studied mechanism in analytical form, the dynamic model is established by using the assumed mode method. To select the mode shape function, the software of multi-body system dynamics is used to simulate the dynamic behavior of the mechanism, and then the data-driven method which combines the DMD and SINDy algorithms is employed to identify the reasonable mode shape functions for the flexible links. To suppress the vibration of the flexible links, a state observer for the end-effector is constructed by a neural network, and the model-based control law is designed on this basis. In comparison with the model-free controller, the proposed controller with developed dynamic model has promising performance in terms of tracking accuracy and vibration suppression.

Deep learning models have revolutionized various fields, from image recognition to natural language processing, by achieving unprecedented levels of accuracy. However, their increasing energy consumption has raised concerns about their environmental impact, disadvantaging smaller entities in research and exacerbating global energy consumption. In this paper, we explore the trade-off between model accuracy and electricity consumption, proposing a metric that penalizes large consumption of electricity. We conduct a comprehensive study on the electricity consumption of various deep learning models across different GPUs, presenting a detailed analysis of their accuracy-efficiency trade-offs. By evaluating accuracy per unit of electricity consumed, we demonstrate how smaller, more energy-efficient models can significantly expedite research while mitigating environmental concerns. Our results highlight the potential for a more sustainable approach to deep learning, emphasizing the importance of optimizing models for efficiency. This research also contributes to a more equitable research landscape, where smaller entities can compete effectively with larger counterparts. This advocates for the adoption of efficient deep learning practices to reduce electricity consumption, safeguarding the environment for future generations whilst also helping ensure a fairer competitive landscape. Deep learning has emerged as a powerful technology, achieving remarkable breakthroughs across various domains. From image recognition and natural language processing to autonomous driving and healthcare diagnostics, deep learning models have redefined the boundaries of what machines can accomplish. The stunning advances in accuracy have transformed industries, offering new solutions to long-standing problems.

Symbolic regression (SR) describes the task of finding a symbolic function that accurately represents the connection between given input and output data. At the same time, the function should be as simple as possible to ensure robustness against noise and interpretability. This is of particular interest for applications where the aim is to (mathematically) analyze the resulting function afterward or get further insights into the process to ensure trustworthiness, for instance, in physical or chemical sciences (Quade et al., 2016; Angelis et al., 2023; Wang et al., 2019). The range of possible applications of SR is therefore vast, from predicting the dynamics of ecosystems (Chen et al., 2019), forecasting the solar power for energy production (Quade et al., 2016), estimating the development of financial markets (Liu and Guo, 2023), analyzing the stability of certain materials (He and Zhang, 2021) to planning optimal trajectories for robots (Oplatkova and Zelinka, 2007), to name but a few. Moreover, as Angelis et al. (2023) points out, the number of papers on SR has increased significantly in recent years, highlighting the relevance and research interest in this area. SR is a specific regression task in machine learning that aims to find an accurate model without any assumption by the user related to the specific data set.

Normalising Flows are non-parametric statistical models characterised by their dual capabilities of density estimation and generation. This duality requires an inherently invertible architecture. However, the requirement of invertibility imposes constraints on their expressiveness, necessitating a large number of parameters and innovative architectural designs to achieve good results. Whilst flow-based models predominantly rely on neural-network-based transformations for expressive designs, alternative transformation methods have received limited attention. In this work, we present Ferumal flow, a novel kernelised normalising flow paradigm that integrates kernels into the framework. Our results demonstrate that a kernelised flow can yield competitive or superior results compared to neural network-based flows whilst maintaining parameter efficiency. Kernelised flows excel especially in the low-data regime, enabling flexible non-parametric density estimation in applications with sparse data availability.

Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. Classical methods for AMR depend on heuristics or expensive error estimators, hindering their use for complex simulations. Recent learning-based AMR methods tackle these issues, but so far scale only to simple toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents. This framework allows for a spatial reward formulation that simplifies the credit assignment problem, which we combine with Message Passing Networks to propagate information between neighboring mesh elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference. Additionally, it achieves a speedup of up to $2$ orders of magnitude compared to uniform refinements in more demanding simulations. We outperform learned baselines and heuristics, achieving a refinement quality that is on par with costly error-based oracle AMR strategies.

The classical Model Predictive Path Integral (MPPI) control framework lacks reliable safety guarantees since it relies on a risk-neutral trajectory evaluation technique, which can present challenges for safety-critical applications such as autonomous driving. Additionally, if the majority of MPPI sampled trajectories concentrate in high-cost regions, it may generate an infeasible control sequence. To address this challenge, we propose the U-MPPI control strategy, a novel methodology that can effectively manage system uncertainties while integrating a more efficient trajectory sampling strategy. The core concept is to leverage the Unscented Transform (UT) to propagate not only the mean but also the covariance of the system dynamics, going beyond the traditional MPPI method. As a result, it introduces a novel and more efficient trajectory sampling strategy, significantly enhancing state-space exploration and ultimately reducing the risk of being trapped in local minima. Furthermore, by leveraging the uncertainty information provided by UT, we incorporate a risk-sensitive cost function that explicitly accounts for risk or uncertainty throughout the trajectory evaluation process, resulting in a more resilient control system capable of handling uncertain conditions. By conducting extensive simulations of 2D aggressive autonomous navigation in both known and unknown cluttered environments, we verify the efficiency and robustness of our proposed U-MPPI control strategy compared to the baseline MPPI. We further validate the practicality of U-MPPI through real-world demonstrations in unknown cluttered environments, showcasing its superior ability to incorporate both the UT and local costmap into the optimization problem without introducing additional complexity.

We present a novel unsupervised machine-learning sock sensor based on Gaussian Mixture Models (GMMs). The proposed GMM sensor demonstrates remarkable accuracy in detecting shocks and is robust across diverse test cases with significantly less parameter tuning than other options. We compare the GMM-based sensor with state-of-the-art alternatives. All methods are integrated into a high-order compressible discontinuous Galerkin solver, where two stabilization approaches are coupled to the sensor to provide examples of possible applications. The Sedov blast and double Mach reflection cases demonstrate that our proposed sensor can enhance hybrid sub-cell flux-differencing formulations by providing accurate information of the nodes that require low-order blending. Besides, supersonic test cases including high Reynolds numbers showcase the sensor performance when used to introduce entropy-stable artificial viscosity to capture shocks, demonstrating the same effectiveness as fine-tuned state-of-the-art sensors. The adaptive nature and ability to function without extensive training datasets make this GMM-based sensor suitable for complex geometries and varied flow configurations. Our study reveals the potential of unsupervised machine-learning methods, exemplified by this GMM sensor, to improve the robustness and efficiency of advanced CFD codes.

Backward propagation (BP) is widely used to compute the gradients in neural network training. However, it is hard to implement BP on edge devices due to the lack of hardware and software resources to support automatic differentiation. This has tremendously increased the design complexity and time-to-market of on-device training accelerators. This paper presents a completely BP-free framework that only requires forward propagation to train realistic neural networks. Our technical contributions are three-fold. Firstly, we present a tensor-compressed variance reduction approach to greatly improve the scalability of zeroth-order (ZO) optimization, making it feasible to handle a network size that is beyond the capability of previous ZO approaches. Secondly, we present a hybrid gradient evaluation approach to improve the efficiency of ZO training. Finally, we extend our BP-free training framework to physics-informed neural networks (PINNs) by proposing a sparse-grid approach to estimate the derivatives in the loss function without using BP. Our BP-free training only loses little accuracy on the MNIST dataset compared with standard first-order training. We also demonstrate successful results in training a PINN for solving a 20-dim Hamiltonian-Jacobi-Bellman PDE. This memory-efficient and BP-free approach may serve as a foundation for the near-future on-device training on many resource-constraint platforms (e.g., FPGA, ASIC, micro-controllers, and photonic chips).

While hyperparameter tuning shows theoretical promise, its practical efficacy is not universally superior, as evidenced in Figure 4. The results thus offer a balanced perspective that marries theoretical rigor with empirical validation, fulfilling both academic and practical requirements. Implications for the Field of Machine Learning and Predictive Modeling Robustness and Generalization: The ensemble and stacking methods offer a mathematically substantiated pathway to improve the generalization capabilities of predictive models. Interpretability: The feature integration techniques not only improve model performance but also offer better interpretability by highlighting important features through mathematical formulations. Optimization: The proven convergence of Bayesian Optimization to the global optimum has far-reaching implications for hyperparameter tuning in models, as formalized by the EI equation. Unified Framework: This research provides a unified, mathematically rigorous framework for integrating Bayesian and non-Bayesian approaches, thereby setting a new benchmark for hybrid predictive systems. Future Research Directions Scalability: Investigating the scalability of the proposed methods, particularly in the context of the ensemble and Bayesian optimization equations, for larger datasets and more models. Real-world Applications: Extending this research to specific domains like healthcare, finance, and natural language processing to assess the practical utility of the proposed methods. Advanced Optimization Techniques: Exploring other optimization techniques that could further improve the efficiency and effectiveness of the proposed hybrid models, perhaps by introducing new mathematical formulations.