In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD) reinforcement learning methods can be formally derived, not by starting from their original objective functions, as previously attempted, but rather from a primal-dual saddle-point objective function. We also conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Previous analyses of this class of algorithms use stochastic approximation techniques to prove asymptotic convergence, and do not provide any finite-sample analysis. We also propose an accelerated algorithm, called GTD2-MP, that uses proximal "mirror maps" to yield an improved convergence rate. The results of our theoretical analysis imply that the GTD family of algorithms are comparable and may indeed be preferred over existing least squares TD methods for off-policy learning, due to their linear complexity. We provide experimental results showing the improved performance of our accelerated gradient TD methods.

Scientists have found a way to boost the battery life of your smartphone or tablet by several hours. They claim that shifting reams of data from your favourite apps to cloud storage services could cut the power they burn through by up to 60 per cent. Tools would identify the most power-hungry parts of a mobile app and then move them to the cloud using a technique called code-offloading. For an average smartphone with an average battery capacity, applying the technology to every app could extend the battery life by up to six hours in a'best-case scenario', researchers said. The technology could be used to build the next generation of disaster relief or search and rescue robots, where battery life is critical.

This post was done in partnership with Wirecutter. When readers choose to buy Wirecutter's independently chosen editorial picks, Wirecutter and Engadget may earn affiliate commission. Although flying a drone might sound like the biggest risk in operating one, dealing with the batteries is potentially more explosive. At the 100 hospital emergency rooms that report electronics-related injury cases to the US Consumer Product Safety Commission, more than 200 incidents (PDF) involving drone batteries, stemming from fire, smoke, and explosions, were recorded between 2012 and 2017. Not every drone-battery incident results in an injury, but each pilot and expert I interviewed had a story about an exploding or fiery lithium battery going off especially after it had repeatedly crashed to the ground inside a drone.

"Ladies and gentlemen, I'll be very glad to tell my story. I am a smart fellow as I have a very fine brain of 48 electrical relays." This is how Elektro the robot introduced itself to crowds at the 1939 New York World's Fair. Standing 2.1 meters tall and weighing 118 kilograms, Elektro performed 26 different tricks, including walking, talking, counting, and singing. It had a vocabulary of approximately 700 words, although its responses were all prerecorded and played back from 33⅓-rpm records.

Ambient backscatter has been introduced with a wide range of applications for low power wireless communications. In this article, we propose an optimal and low-complexity dynamic spectrum access framework for RFpowered ambient backscatter system. Under the dynamics of the ambient signals, we first adopt the Markov decision process (MDP) framework to obtain the optimal policy for the secondary transmitter, aiming to maximize the system throughput. However, the MDP-based optimization requires complete knowledge of environment parameters, e.g., the probability of a channel to be idle and the probability of a successful packet transmission, that may not be practical to obtain. To cope with such incomplete knowledge of the environment, we develop a low-complexity online reinforcement learning algorithm that allows the secondary transmitter to "learn" from its decisions and then attain the optimal policy. Simulation results show that the proposed learning algorithm not only efficiently deals with the dynamics of the environment, but also improves the average throughput up to 50% and reduces the blocking probability and delay up to 80% compared with conventional methods. Dynamic spectrum access (DSA) has been considered as a promising solution to improve the utilization of radio spectrum [2]. As DSA standard frameworks, the Federal Communications Commission and the European Telecommunications Standardization Institute have recently proposed Spectrum Access Systems (SAS) and Licensed Shared Access (LSA) respectively [3]. In both SAS and LSA, spectrum users are prioritized at different levels/tiers (e.g., there are three types of users with a decreasing order of priority: Incumbent Users (IUs), Priority Access Licensees (PALs), and General Authorized Access (GAAs)). Without loss of generality, in this work, we refer users with higher priority as IUs and users with lower priority as secondary users (SUs). DSA harvests under-utilized spectrum chunks by allowing an SU to dynamically access (temporarily) idle spectrum bands/whitespaces to transmit data.

I started out by just wanting to make a very small drone. But I quickly realized that there was a limiting factor in just how small and light I could make any design: the motors. Even small motors are still discrete packages that have to be attached to all the other electronic and structural elements. So I began wondering if there was a way to merge these elements and save some mass. I drew inspiration from how some radio systems used antennas made from the copper traces on a printed circuit board (PCB).

Even as STEM toys become more popular, Lego's construction sets remain the gold standard, with more recent products like Mindstorms and Boost expanding into the fields of robotics and coding. Now bread-and-butter sets like Lego city and licensed products like DC Superheroes get nifty additions like motorized parts and app connectivity, starting with Batman's iconic vehicle. This is the Batmobile from the current slate of DCEU films, complete with tiny cannons and a scowling Batman minifig. For the most part it's a pretty normal Lego set where you follow the instructions and snap it together with your hands. But maybe it was a little too normal: I still had to rely on a paper manual for help, as there are no instructions in the app. Maybe I'm a little spoiled at this point by products like Kamigami and Labo, but one of the advantages of making something app connected is the ability to have interactive diagrams that animate where to place each piece and even allow you to rotate the image for a better look.

The one thing I know from going to the Consumer Electronics Show over the past few years is that every time you go, you will find at least one Chinese company that you've never heard of … and that makes just about every kind of electronics gadget you can imagine. This year that one Chinese company was Haier. If the name sounds vaguely familiar, that's because Haier bought General Electric's appliance business in 2016. And if it doesn't, that's because the company hasn't really announced itself under its own brand widely. I did a walk-through of the company's booth, and it's worth skimming through despite the extremely authentic CES chaos, bustle, and noise.

The discovery of new materials can bring enormous societal and technological progress. In this context, exploring completely the large space of potential materials is computationally intractable. Here, we review methods for achieving inverse design, which aims to discover tailored materials from the starting point of a particular desired functionality. Recent advances from the rapidly growing field of artificial intelligence, mostly from the subfield of machine learning, have resulted in a fertile exchange of ideas, where approaches to inverse molecular design are being proposed and employed at a rapid pace. Among these, deep generative models have been applied to numerous classes of materials: rational design of prospective drugs, synthetic routes to organic compounds, and optimization of photovoltaics and redox flow batteries, as well as a variety of other solid-state materials.

The use of natural wood, hand-knitted fabrics, and leather lend a rustic hand-made vibe to the Maverick.Cavalier Cavalier Audio is a New Jersey-based designer and manufacturer of voice-controlled speakers. The company's latest product is Maverick portable Bluetooth and Wi-Fi speaker with Amazon Alexa voice control. Maverick blends a hand-crafted finish and immersive sound to a market that's positively crowded out with'me-too' products produced from mass-produced plastics and so-so design. The Maverick has been designed to stand out from the crowd by using authentic materials, stylish design while incorporating the latest audio technology. Maverick was dreamed up by a team of musicians and engineers with the aim of offering world-class acoustics to life via a 20W stereo speaker system featuring two active drivers and dual passive radiators that the designers claim will produce true room-filling experience.