We consider a multi-agent episodic MDP setup where an agent (leader) takes action at each step of the episode followed by another agent (follower). The state evolution and rewards depend on the joint action pair of the leader and the follower. Such type of interactions can find applications in many domains such as smart grids, mechanism design, security, and policymaking. We are interested in how to learn policies for both the players with provable performance guarantee under a bandit feedback setting. We focus on a setup where both the leader and followers are {\em non-myopic}, i.e., they both seek to maximize their rewards over the entire episode and consider a linear MDP which can model continuous state-space which is very common in many RL applications. We propose a {\em model-free} RL algorithm and show that $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ regret bounds can be achieved for both the leader and the follower, where $d$ is the dimension of the feature mapping, $H$ is the length of the episode, and $T$ is the total number of steps under the bandit feedback information setup. Thus, our result holds even when the number of states becomes infinite. The algorithm relies on {\em novel} adaptation of the LSVI-UCB algorithm. Specifically, we replace the standard greedy policy (as the best response) with the soft-max policy for both the leader and the follower. This turns out to be key in establishing uniform concentration bound for the value functions. To the best of our knowledge, this is the first sub-linear regret bound guarantee for the Markov games with non-myopic followers with function approximation.

Efficient energy consumption is crucial for achieving sustainable energy goals in the era of climate change and grid modernization. Thus, it is vital to understand how energy is consumed at finer resolutions such as household in order to plan demand-response events or analyze impacts of weather, electricity prices, electric vehicles, solar, and occupancy schedules on energy consumption. However, availability and access to detailed energy-use data, which would enable detailed studies, has been rare. In this paper, we release a unique, large-scale, digital-twin of residential energy-use dataset for the residential sector across the contiguous United States covering millions of households. The data comprise of hourly energy use profiles for synthetic households, disaggregated into Thermostatically Controlled Loads (TCL) and appliance use. The underlying framework is constructed using a bottom-up approach. Diverse open-source surveys and first principles models are used for end-use modeling. Extensive validation of the synthetic dataset has been conducted through comparisons with reported energy-use data. We present a detailed, open, high resolution, residential energy-use dataset for the United States.

SAN FRANCISCO, Dec 13 (Reuters) - Alphabet Inc's (GOOGL.O) Waymo has applied for the final permit it needs in California before it can sell fully autonomous rides, the company told Reuters on Tuesday. A decision on its application, which was submitted Monday to the California Public Utilities Commission, could take months. General Motors Co's (GM.N) Cruise is the only company with the permit so far and has charged for driverless rides in San Francisco since June. The two rivals are frontrunners in the slow-moving effort to demonstrate that autonomous transport can become a widely available and profitable service, with San Francisco's hills, weather and clogged roads making it a key proving ground. GM plans to expand to more cities next year.

Fox News correspondent Jeff Paul has the latest from Kyiv, Ukraine, on'America Reports.' Russia launched 13 Iranian-made Shahed drones into Kyiv Wednesday, in one of the largest attempted strikes of its kind on the Ukraine capital, Fox News Digital was told. All 13 drones were apparently targeting energy infrastructure but were downed before they reached their targets and no military or civilian casualties have been reported, city administrative and defense officials said. Air raid sirens rang out shortly before 6 a.m. Rescuers and police experts examine remains of a drone following a strike on an administrative building in the Ukrainian capital Kyiv on Dec. 14, 2022. One source told Fox News Digital that Russia was looking to hit the city's energy infrastructure and in a Telegram post city officials said, "once again [Russia] targeted the critical infrastructure of the region and the capital."

At the intersection of machine learning and energy consumption stands an incredibly powerful force with the potential to transform the way we globally produce and consume energy. So powerful in fact, that the concept of merging machine learning and renewable resources has been named the "energy internet" by economic theorist and author Jeremy Rifkin or "digital efficiency" by Intel and GE . Going green with machine learning solutions can drastically improve the way we consume energy, in terms of lower operational costs, more efficient production, better use of natural resources and lower environmental impacts. Before energy reaches the grid, machine learning has the ability to revolutionize the way it is collected. GE, other energy and heavy industry leaders, and machine learning research groups have been working on addressing these challenges, yet implementing changes across highly centralized networks is no small task.

Welcome to insideBIGDATA's annual technology predictions round-up! The big data industry has significant inertia moving into 2023. In order to give our valued readers a pulse on important new trends leading into next year, we here at insideBIGDATA heard from all our friends across the vendor ecosystem to get their insights, reflections and predictions for what may be coming. We were very encouraged to hear such exciting perspectives. Even if only half actually come true, Big Data in the next year is destined to be quite an exciting ride. There are many reasons why a customer would choose to implement their architecture on multiple clouds whether it's technology, market, or business-driven. When this happens, many times this leads to transactional and operational data being stored on multiple cloud platforms. The challenge this brings is how to gain insight into these without resorting to implementing multiple disparate data platforms. Historically data virtualization tools have been ...

Research led by Monash University, RMIT and the University of Adelaide has developed an accurate method of controlling optical circuits on fingernail-sized photonic integrated circuits. The development, published in the prestigious international journal Optica builds on the work by the same team who recently created the world's first self-calibrated photonic chip. Photonics, or the use of light particles to store and transmit information, is a burgeoning field, supporting our need to create faster, better, more efficient and more sustainable technology. Programmable photonic integrated circuits (PICs), offer diverse signal processing functions within a single chip, and present promising solutions for applications ranging from optical communications to artificial intelligence. Whether it's downloading movies or keeping a satellite on course, photonics is radically changing the way we live, revolutionising the processing capability of large scale equipment onto a chip the size of a human fingernail.

Spatial audio methods are gaining a growing interest due to the spread of immersive audio experiences and applications, such as virtual and augmented reality. For these purposes, 3D audio signals are often acquired through arrays of Ambisonics microphones, each comprising four capsules that decompose the sound field in spherical harmonics. In this paper, we propose a dual quaternion representation of the spatial sound field acquired through an array of two First Order Ambisonics (FOA) microphones. The audio signals are encapsulated in a dual quaternion that leverages quaternion algebra properties to exploit correlations among them. This augmented representation with 6 degrees of freedom (6DOF) involves a more accurate coverage of the sound field, resulting in a more precise sound localization and a more immersive audio experience. We evaluate our approach on a sound event localization and detection (SELD) benchmark. We show that our dual quaternion SELD model with temporal convolution blocks (DualQSELD-TCN) achieves better results with respect to real and quaternion-valued baselines thanks to our augmented representation of the sound field. Full code is available at: https://github.com/ispamm/DualQSELD-TCN.

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.

In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.