This paper studies distributed adaptive estimation over sensor networks with partially known source dynamics. We present parallel continuous-time and discrete-time designs in which each node runs a local adaptive observer and exchanges information over a directed graph. For both time scales, we establish stability of the network coupling operators, prove boundedness of all internal signals, and show convergence of each node estimate to the source despite model uncertainty and disturbances. We further derive input-to-state stability (ISS) bounds that quantify robustness to bounded process noise. A key distinction is that the discrete-time design uses constant adaptive gains and per-step regressor normalization to handle sampling effects, whereas the continuous-time design does not. A unified Lyapunov framework links local observer dynamics with graph topology. Simulations on star, cyclic, and path networks corroborate the analysis, demonstrating accurate tracking, robustness, and scalability with the number of sensing nodes.

A.1 Central Limit Theorem for SA Statements in this part are adapted from [4, Chapter 2 and 3]. We can now state the central limit theorem. To guarantee Assumption 2 and Assumption 3, we make the following assumption. Q-learning even when equipped with neural networks [33]. The proof of this lemma is deferred to the next section.

We thank all the reviewers for their careful feedback and will revise our paper accordingly. Such a fact is presented in the classic paper "An analysis of temporal-difference learning with function Similar facts can be found for other TD algorithms (e.g. Reviewer 1 is correct in that a discount factor is needed. Now we address specific reviewer comments below. A reference for this is the classic paper "An Finally, the "-" sign in Line 213 is due to the Hurwtiz assumption.

This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued that it is not possible in general to obtain a Hoeffding bound on the error sequence, even when the underlying Markov chain is reversible and geometrically ergodic, such as the M/M/1 queue. This is motivation for the focus on mean square error bounds for parameter estimates. It is shown that mean square error achieves the optimal rate of $O(1/n)$, subject to conditions on the step-size sequence. Moreover, the exact constants in the rate are obtained, which is of great value in algorithm design.

Machine learning/AI capability is increasingly transforming how we engage with technology. Just look at how mainstream digital voice assistants such as Alexa and Siri, customer-service chat bots and bank-fraud detection tools have become. I see promise in healthcare using machine learning/AI technology. One example is using AI to detect breast cancer by analyzing mammograms. Leveraging the expertise of subject matter experts in healthcare is crucial to successfully applying AI to solving healthcare challenges.

The use of target networks has been a popular and key component of recent deep Q-learning algorithms for reinforcement learning, yet little is known from the theory side. In this work, we introduce a new family of target-based temporal difference (TD) learning algorithms and provide theoretical analysis on their convergences. In contrast to the standard TD-learning, target-based TD algorithms maintain two separate learning parameters-the target variable and online variable. Particularly, we introduce three members in the family, called the averaging TD, double TD, and periodic TD, where the target variable is updated through an averaging, symmetric, or periodic fashion, mirroring those techniques used in deep Q-learning practice. We establish asymptotic convergence analyses for both averaging TD and double TD and a finite sample analysis for periodic TD. In addition, we also provide some simulation results showing potentially superior convergence of these target-based TD algorithms compared to the standard TD-learning. While this work focuses on linear function approximation and policy evaluation setting, we consider this as a meaningful step towards the theoretical understanding of deep Q-learning variants with target networks.

This is the fourth in a continuing series of stories previewing sessions of importance to cloud application developers at the Cloud Expo conference, which takes place June 7 to 9 at the Jacob Javits Center in New York. Judith Hurwitz is president and CEO of Hurwitz & Associates, a Needham, Mass., research and consulting firm focused on emerging technology, including big data, cognitive computing and governance. She is co-author of the book Cognitive Computing and Big Data Analytics, published in 2015. Her Cloud Expo session, "What Is the Business Imperative for Cognitive Computing?" is scheduled for Wednesday, June 8, at 8:40 a.m. In it, she puts cognitive computing into perspective with its value to the business, examines what it takes to build a cognitive application and identifies the types of services that best fit this data-driven approach.

Furthermore, traveling in any direction orthogonal to the gradient maintains the value of the function. In this work, we show that these orthogonal directions that are ignored by gradient descent can be critical in equilibrium problems. Equilibrium problems have drawn heightened attention in machine learning due to the emergence of the Generative Adversarial Network (GAN). We use the framework of Variational Inequalities to analyze popular training algorithms for a fundamental GAN variant: the Wasserstein Linear-Quadratic GAN. We show that the steepest descent direction causes divergence from the equilibrium, and guaranteed convergence to the equilibrium is achieved through following a particular orthogonal direction. We call this successful technique Crossing-the-Curl, named for its mathematical derivation as well as its intuition: identify the game's axis of rotation and move "across" space in the direction towards smaller "curling".