Reinforcement learning algorithms typically require too many trial-and-error' experiences before reaching a desirable behaviour. A considerable amount of ongoing research is focused on speeding up this learning process by using external knowledge. We contribute in several ways, proposing novel approaches to transfer learning and learning from demonstration, as well as an ensemble approach to combine knowledge from various sources.

Reinforcement learning describes how a learning agent can achieve optimal behaviour based on interactions with its environment and reward feedback. A limiting factor in reinforcement learning as employed in artificial intelligence is the need for an often prohibitively large number of environment samples before the agent reaches a desirable level of performance. Learning from demonstration is an approach that provides the agent with demonstrations by a supposed expert, from which it should derive suitable behaviour. Yet, one of the challenges of learning from demonstration is that no guarantees can be provided for the quality of the demonstrations, and thus the learned behavior. In this paper, we investigate the intersection of these two approaches, leveraging the theoretical guarantees provided by reinforcement learning, and using expert demonstrations to speed up this learning by biasing exploration through a process called reward shaping. This approach allows us to leverage human input without making an erroneous assumption regarding demonstration optimality. We show experimentally that this approach requires significantly fewer demonstrations, is more robust against suboptimality of demonstrations, and achieves much faster learning than the recently developed HAT algorithm.

Three years ago, Customs and Border Protection placed an order for self-flying aircraft that could launch on their own, rendezvous, locate and monitor multiple targets on the ground without any human intervention. In its reasoning for the order, CBP said the level of monitoring required to secure America's long land borders from the sky was too cumbersome for people alone. To research and build the drones, CBP handed $500,000 to Mitre Corp., a trusted nonprofit Skunk Works that was already furnishing border police with prototype rapid DNA testing and smartwatch hacking technology. They were "tested but not fielded operationally" as "the gap from simulation to reality turned out to be much larger than the research team originally envisioned," a CBP spokesperson says. This year, America's border police will test automated drones from Skydio, the Redwood City, Calif.-based startup that on Monday announced it had raised an additional $170 million in venture funding at a valuation of $1 billion. That brings the total raised for Skydio to $340 million.

Three years ago, Customs and Border Protection placed an order for self-flying aircraft that could launch on their own, rendezvous, locate and monitor multiple targets on the ground without any human intervention. In its reasoning for the order, CBP said the level of monitoring required to secure America's long land borders from the sky was too cumbersome for people alone. To research and build the drones, CBP handed $500,000 to Mitre Corp., a trusted nonprofit Skunk Works that was already furnishing border police with prototype rapid DNA testing and smartwatch hacking technology. They were "tested but not fielded operationally" as "the gap from simulation to reality turned out to be much larger than the research team originally envisioned," a CBP spokesperson says. This year, America's border police will test automated drones from Skydio, the Redwood City, Calif.-based startup that on Monday announced it had raised an additional $170 million in venture funding at a valuation of $1 billion. That brings the total raised for Skydio to $340 million.

Three years ago, Customs and Border Protection placed an order for self-flying aircraft that could launch on their own, rendezvous, locate and monitor multiple targets on the ground without any human intervention. In its reasoning for the order, CBP said the level of monitoring required to secure America's long land borders from the sky was too cumbersome for people alone. To research and build the drones, CBP handed $500,000 to Mitre Corp., a trusted nonprofit Skunk Works that was already furnishing border police with prototype rapid DNA testing and smartwatch hacking technology. They were "tested but not fielded operationally" as "the gap from simulation to reality turned out to be much larger than the research team originally envisioned," a CBP spokesperson says. This year, America's border police will test automated drones from Skydio, the Redwood City, Calif.-based startup that on Monday announced it had raised an additional $170 million in venture funding at a valuation of $1 billion.

Skydio makes one of the most incredible drones on the market, and while we haven't gotten to review the new Skydio 2 yet, the tiny California startup is already setting its ambitions higher than prosumers and videographers. For industrial and commercial entities, it wants to remove humans from the equation entirely, letting them rely on its obstacle-dodging, self-flying technology for automated mapping and surveillance. To that end, it's announcing the Skydio 2 Dock, a drone-in-a-box solution that theoretically lets the Skydio 2 fly mission after mission all by itself. As you can see in the video above, it's got a motorized door and slide-out arm that the drone can land on as well as a built-in charging station for a special version of the Skydio 2's battery with contact pins on the bottom. Skydio co-founder Adam Bry tells me that his drone can find its way back to the box without GPS, thanks to its visual and inertial navigation systems, and it can land precisely on that pad time after time, thanks to a pair of visual markers on top.

It's 2035, the Second American Civil War has been won by the other side, and you find yourself in a heap of trouble with Attorney General Logan Paul. He has dispatched an all-seeing eye-in-the-sky to tail you, an agile flying machine equipped with 13 cameras and a top speed of 25 miles per hour. The drone knows your face, your gait and your clothing. It hovers persistently behind your back, moving when you move, stopping when you stop, resisting every effort to shake it. You run into the woods, but you still can't lose it.

Almost two years ago, a startup called Skydio posted some video of a weird-looking drone autonomously following people as they jogged and biked along paths and around trees. Even without much in the way of detail, this was exciting for three reasons: First, the drone was moving at a useful speed and not crashing into stuff using only onboard sensing and computing, and second, the folks behind Skydio included Adam Bry and Abe Bachrach, who worked on high-speed autonomous flight at MIT before cofounding Project Wing at Google[x] (now just called X). The third reason we were excited about Skydio's drone was that, as much as it looked like a research project, it was actually designed to be commercialized, and today, Skydio is (finally!) And before you think that you've seen flying cameras before, we promise you've never seen anything like the R1: as Bry told us two years ago, Skydio's goal was "to provide a trustworthy and magical experience." Initially, Skydio sent us a couple different videos to show off the new R1.