A critical element in the development and deployment of AVs is the design of car-following (CF) controllers capable of producing desirable performance in real-world settings. Ideally, a CF control system would effectively and safely handle the longitudinal maneuvers of the vehicle at every encounter it faces. However, designing and training such a controller requires enormous data, testing, and experimentation that covers all possible driving scenarios/encounters. In other words, it requires us to have a perfect understanding of the environment these AVs would be operating under. Clearly, this is very challenging and, possibly, unattainable. AVs are likely to encounter unseen scenarios and be exposed to exogenous and endogenous uncertainties in the physical world. The sources of exogenous and endogenous uncertainties are vast and roughly classified into (Macfarlane and Stroila, 2016; Yao et al., 2020; Katrakazas et al., 2015): (i) vehicular and system dynamics (e.g., vehicle condition, road gradient, aerodynamic drag force, external loads, transmission, brake, the performance of the engine, etc.), (ii) environmental conditions (snow, dust, wind, wet conditions, etc.), and (iii) situational detection (e.g., sensor/measurement errors, radar errors, vehicle speed fluctuations, vehicle localization, communication latency, etc.). All these types of uncertainties can hinder desirable performance (e.g., stability). Yet, a major challenge lies in the complexity of integrating these uncertainties into the control system and the design of the AV.

Estimating causal queries, such as changes in protein abundance in response to a perturbation, is a fundamental task in the analysis of biomolecular pathways. The estimation requires experimental measurements on the pathway components. However, in practice many pathway components are left unobserved (latent) because they are either unknown, or difficult to measure. Latent variable models (LVMs) are well-suited for such estimation. Unfortunately, LVM-based estimation of causal queries can be inaccurate when parameters of the latent variables are not uniquely identified, or when the number of latent variables is misspecified. This has limited the use of LVMs for causal inference in biomolecular pathways. In this manuscript, we propose a general and practical approach for LVM-based estimation of causal queries. We prove that, despite the challenges above, LVM-based estimators of causal queries are accurate if the queries are identifiable according to Pearl's do-calculus, and describe an algorithm for its estimation. We illustrate the breadth and the practical utility of this approach for estimating causal queries in four synthetic and two experimental case studies, where structures of biomolecular pathways challenge the existing methods for causal query estimation. The code and the data documenting all the case studies are available at \url{https://github.com/srtaheri/LVMwithDoCalculus}

When sampling for Bayesian inference, one popular approach is to use Hamiltonian Monte Carlo (HMC) and specifically the No-U-Turn Sampler (NUTS) which automatically decides the end time of the Hamiltonian trajectory. However, HMC and NUTS can require numerous numerical gradients of the target density, and can prove slow in practice. We propose Hamiltonian neural networks (HNNs) with HMC and NUTS for solving Bayesian inference problems. Once trained, HNNs do not require numerical gradients of the target density during sampling. Moreover, they satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well-suited for use within HMC and NUTS because stationarity can be shown. We also propose an HNN extension called latent HNNs (L-HNNs), which are capable of predicting latent variable outputs. Compared to HNNs, L-HNNs offer improved expressivity and reduced integration errors. Finally, we employ L-HNNs in NUTS with an online error monitoring scheme to prevent sample degeneracy in regions of low probability density. We demonstrate L-HNNs in NUTS with online error monitoring on several examples involving complex, heavy-tailed, and high-local-curvature probability densities. Overall, L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. Compared to traditional NUTS, L-HNNs in NUTS with online error monitoring required 1--2 orders of magnitude fewer numerical gradients of the target density and improved the effective sample size (ESS) per gradient by an order of magnitude.

Abstract-- Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from Figure 1: Our active exploration strategy evaluated on a challenging scratch with our method, on a challenging ball pushing task Tilted Pushing task in simulation (left) and on the real robot (right). Our real-world robot experiment serves as sparse reward model in order to bring the ball to a target location. We believe that for of dexterous manipulation capabilities was one of the major robots to reach human-level manipulation skills, they must driving factors in the development of the human mind [1] actively explore and adapt to new instances of a task. Performing manipulation is cognitively highly demanding, forcing the We define active exploration as the directed search of the agent to reason not only about the impact of its actions on agent, during the learning process, for unvisited state-action itself, but also on the environment. This inherent complexity pairs that would maximize the agent's performance.

We study the problem of learning generalized linear models under adversarial corruptions. We analyze a classical heuristic called the iterative trimmed maximum likelihood estimator which is known to be effective against label corruptions in practice. Under label corruptions, we prove that this simple estimator achieves minimax near-optimal risk on a wide range of generalized linear models, including Gaussian regression, Poisson regression and Binomial regression. Finally, we extend the estimator to the more challenging setting of label and covariate corruptions and demonstrate its robustness and optimality in that setting as well.

Each machine learning algorithm handles one specific problem, and this way beginners can dive into one of these to figure out solutions, one at a time. Here is a compilation of the top machine learning algorithms that are frequently used in all machine learning fields. Now, you can practice ML algorithms here. Forming relationships between two variables is almost the starting point of a model, and linear regression in machine learning achieves that. The relationship between the dependent and independent variables is established by aligning them on a regression line.

Maximum likelihood estimation (MLE) is the predominant algorithm for training text generation models. This paradigm relies on direct supervision examples, which is not applicable to many emerging applications, such as generating adversarial attacks or generating prompts to control language models. Reinforcement learning (RL) on the other hand offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. Yet previous RL algorithms for text generation, such as policy gradient (on-policy RL) and Q-learning (off-policy RL), are often notoriously inefficient or unstable to train due to the large sequence space and the sparse reward received only at the end of sequences. In this paper, we introduce a new RL formulation for text generation from the soft Q-learning (SQL) perspective. It enables us to draw from the latest RL advances, such as path consistency learning, to combine the best of on-/off-policy updates, and learn effectively from sparse reward. We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation. Experiments show our approach consistently outperforms both task-specialized algorithms and the previous RL methods.

We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments. We propose a simple algorithm which combines kernelized IV methods and an arbitrary, adaptive regression algorithm, accessed as a black box. Our algorithm enjoys faster-rate convergence and adapts to the dimensionality of informative latent features, while avoiding an expensive minimax optimization procedure, which has been necessary to establish similar guarantees. It further brings the benefit of flexible machine learning models to quasi-Bayesian uncertainty quantification, likelihood-based model selection, and model averaging. Simulation studies demonstrate the competitive performance of our method.

Bayesian optimisation (BO) algorithms have shown remarkable success in applications involving expensive black-box functions. Traditionally BO has been set as a sequential decision-making process which estimates the utility of query points via an acquisition function and a prior over functions, such as a Gaussian process. Recently, however, a reformulation of BO via density-ratio estimation (BORE) allowed reinterpreting the acquisition function as a probabilistic binary classifier, removing the need for an explicit prior over functions and increasing scalability. In this paper, we present a theoretical analysis of BORE's regret and an extension of the algorithm with improved uncertainty estimates. We also show that BORE can be naturally extended to a batch optimisation setting by recasting the problem as approximate Bayesian inference. The resulting algorithms come equipped with theoretical performance guarantees and are assessed against other batch and sequential BO baselines in a series of experiments.

An appropriate level of arousal induces positive emotions, and a high arousal potential may provoke negative emotions. To explain the effect of arousal on emotional valence, we propose a novel mathematical framework of arousal potential variations in the dual process of human cognition: automatic and controlled. A suitable mathematical formulation to explain the emotions in the dual process is still absent. Our model associates free energy with arousal potential and its variations to explain emotional valence. Decreasing and increasing free energy consequently induce positive and negative emotions, respectively. We formalize a transition from the automatic to the controlled process in the dual process as a change of Bayesian prior. Further, we model emotional valence using free energy increase (FI) when one tries changing one's Bayesian prior and its reduction (FR) when one succeeds in recognizing the same stimuli with a changed prior and define three emotions: "interest," "confusion," and "boredom" using the variations. The results of our mathematical analysis comparing various Gaussian model parameters reveals the following: 1) prediction error (PR) increases FR (representing "interest") when the first prior variance is greater than the second prior variance, 2) PR decreases FR when the first prior variance is less than the second prior variance, and 3) the distance between priors' means always increases FR. We also discuss the association of the outcomes with emotions in the controlled process. The proposed mathematical model provides a general framework for predicting and controlling emotional valence in the dual process that varies with viewpoint and stimuli, as well as for understanding the contradictions in the effects of arousal on the valence.