Drones
Learning Camera Movement Control from Real-World Drone Videos
Hou, Yunzhong, Zheng, Liang, Torr, Philip
This study seeks to automate camera movement control for filming existing subjects into attractive videos, contrasting with the creation of non-existent content by directly generating the pixels. We select drone videos as our test case due to their rich and challenging motion patterns, distinctive viewing angles, and precise controls. Existing AI videography methods struggle with limited appearance diversity in simulation training, high costs of recording expert operations, and difficulties in designing heuristic-based goals to cover all scenarios. To avoid these issues, we propose a scalable method that involves collecting real-world training data to improve diversity, extracting camera trajectories automatically to minimize annotation costs, and training an effective architecture that does not rely on heuristics. Specifically, we collect 99k high-quality trajectories by running 3D reconstruction on online videos, connecting camera poses from consecutive frames to formulate 3D camera paths, and using Kalman filter to identify and remove low-quality data. Moreover, we introduce DVGFormer, an auto-regressive transformer that leverages the camera path and images from all past frames to predict camera movement in the next frame. We evaluate our system across 38 synthetic natural scenes and 7 real city 3D scans. We show that our system effectively learns to perform challenging camera movements such as navigating through obstacles, maintaining low altitude to increase perceived speed, and orbiting towers and buildings, which are very useful for recording high-quality videos. Data and code are available at dvgformer.github.io.
Drift-free Visual SLAM using Digital Twins
Merat, Roxane, Cioffi, Giovanni, Bauersfeld, Leonard, Scaramuzza, Davide
Globally-consistent localization in urban environments is crucial for autonomous systems such as self-driving vehicles and drones, as well as assistive technologies for visually impaired people. Traditional Visual-Inertial Odometry (VIO) and Visual Simultaneous Localization and Mapping (VSLAM) methods, though adequate for local pose estimation, suffer from drift in the long term due to reliance on local sensor data. While GPS counteracts this drift, it is unavailable indoors and often unreliable in urban areas. An alternative is to localize the camera to an existing 3D map using visual-feature matching. This can provide centimeter-level accurate localization but is limited by the visual similarities between the current view and the map. This paper introduces a novel approach that achieves accurate and globally-consistent localization by aligning the sparse 3D point cloud generated by the VIO/VSLAM system to a digital twin using point-to-plane matching; no visual data association is needed. The proposed method provides a 6-DoF global measurement tightly integrated into the VIO/VSLAM system. Experiments run on a high-fidelity GPS simulator and real-world data collected from a drone demonstrate that our approach outperforms state-of-the-art VIO-GPS systems and offers superior robustness against viewpoint changes compared to the state-of-the-art Visual SLAM systems.
See Behind Walls in Real-time Using Aerial Drones and Augmented Reality
Yang, Sikai, Yang, Kang, Chen, Yuning, Zhao, Fan, Du, Wan
This work presents ARD2, a framework that enables real-time through-wall surveillance using two aerial drones and an augmented reality (AR) device. ARD2 consists of two main steps: target direction estimation and contour reconstruction. In the first stage, ARD2 leverages geometric relationships between the drones, the user, and the target to project the target's direction onto the user's AR display. In the second stage, images from the drones are synthesized to reconstruct the target's contour, allowing the user to visualize the target behind walls. Experimental results demonstrate the system's accuracy in both direction estimation and contour reconstruction.
Distributed Intelligent System Architecture for UAV-Assisted Monitoring of Wind Energy Infrastructure
Svystun, Serhii, Melnychenko, Oleksandr, Radiuk, Pavlo, Savenko, Oleg, Lysyi, Andrii
With the rapid development of green energy, the efficiency and reliability of wind turbines are key to sustainable renewable energy production. For that reason, this paper presents a novel intelligent system architecture designed for the dynamic collection and real-time processing of visual data to detect defects in wind turbines. The system employs advanced algorithms within a distributed framework to enhance inspection accuracy and efficiency using unmanned aerial vehicles (UAVs) with integrated visual and thermal sensors. An experimental study conducted at the "Staryi Sambir-1" wind power plant in Ukraine demonstrates the system's effectiveness, showing a significant improvement in defect detection accuracy (up to 94%) and a reduction in inspection time per turbine (down to 1.5 hours) compared to traditional methods. The results show that the proposed intelligent system architecture provides a scalable and reliable solution for wind turbine maintenance, contributing to the durability and performance of renewable energy infrastructure.
Chinese citizen allegedly photographed Vandenberg base with drone, says it was 'probably not a good idea'
Nearly a mile above Vandenberg Space Force Base in Santa Barbara County, a hacked drone soared through restricted airspace for roughly an hour. The lightweight drone photographed sensitive areas of the military facility on Nov. 30, including a complex used by SpaceX, according to federal investigators. The drone then descended back to the ground, where the pilot and another man waited at a nearby park. Four security officers from the military base arrived on the scene and asked the men if they had seen a drone flying through the area, unaware that one of them had tucked the drone under his jacket. Authorities identified that man as 39-year-old Yinpiao Zhou, a Chinese citizen and a lawful permanent resident of the U.S.
Dozens of drones trailed a Coast Guard vessel off New Jersey: US lawmaker
Rep. Chris Smith, R-N.J., opens up about the aerial systems spotted in the Garden State on'The Story.' A U.S. Coast Guard official said one of its vessels was trailed by dozens of drones off the coast of New Jersey recently, according to Rep. Chris Smith, R-N.J. Smith, a guest on "The Story with Martha MacCallum" Tuesday, said he spent Monday night on the beach in Ocean County and spoke to several people, including a U.S. Coast Guard commanding officer stationed in Barnegat Light. Smith learned from the Coast Guard commander that the night before, "one of their 47-foot vessels, boats, was trailed very closely by more than a dozen of these drones." "Now, that to me, is very, very, not just suspicious, provocative, and this could be a foreign power, whether it be [Vladimir] Putin, or it could be Xi Jinping in China, or the Middle East, we can't rule any of that out," the congressman said. Photos taken in the Bay Shore section of Toms River of what appear to be large drones hovering in the area at high altitudes in New Jersey on Sunday, Dec. 8, 2024.
Israeli strikes kill five in southern Lebanon amid shaky ceasefire
At least five people have been killed in Israeli attacks on several towns in southern Lebanon, the country's Health Ministry has said, amid a fragile ceasefire between Israel and Hezbollah. "An Israeli enemy drone strike on the town of Ainata killed one person and wounded another," the ministry said. An "Israeli strike on the town of Bint Jbeil killed three people," while a third "on Beit Lif killed one person", it added. There was no immediate comment from the Israeli military on the attacks. Israel's army escalated its attacks on Lebanon in late September after more than 11 months of cross-border exchanges of fire with the Lebanese armed group Hezbollah, which began firing rockets towards Israel after the Palestinian group Hamas's attack on southern Israel on October 7, 2023.
We need to know whether the drones over New York and New Jersey pose a threat to the homeland
State Sen. John Bramnick joins'Fox & Friends' to discuss the upcoming meeting with Gov. Phil Murphy and officials over mysterious drone sightings in their state. Two years ago, a Chinese balloon the size of three school buses hovered 60,000 feet in the air, drifting across the continental U.S. for seven days. It passed over sensitive security areas, including Malmstrom Air Force Base in Great Falls, Montana, that's home to stockpiles of missiles and nuclear defense infrastructure. Only after it was shot down did we learn this "civilian research airship" that President Biden claimed "was not a major security breach" was communicating with China through an American internet service provider and equipped with thousands of pounds of equipment, including a "massive surveillance payload." One would think the President of the United States and our nation's federal law enforcement agencies would have learned a lesson from this blatant security breach.
Pentagon yet to ask NORTHCOM to intervene against New Jersey drones
Rep. Jeff Van Drew, R-N.J., addresses concerns over mystery drones flying over multiple New Jersey counties and details what sources have told him about their origins. The Pentagon has not yet asked U.S. Northern Command to intervene amid reports of mysterious drones witnessed flying over New Jersey, according to a military spokesperson. The large drone sightings have caused concern and confusion as dozens have been reported and officials are at a loss to explain where they come from. Northern Command confirmed some of the drones have been sighted near U.S. military installations. "We are aware and monitoring the reports of unauthorized drone flights in the vicinity of military installations in New Jersey to include Picatinny Arsenal and Naval Weapons Station Earle, and we refer you to those installations for information on any efforts they are may be conducting to ensure the safety and security of their personnel and operations," a U.S. Nothern Command spokesperson told Fox News Digital.
Latest drone footage captures 'sophisticated' UFOs interacting with each other over New Jersey
The latest footage of bizarre drones in New Jersey captured several craft orbiting each other over Somerset County, while at least 12 counties have reported sightings. The video, released this week, shows three'mystery drones in the air' as two move extremely close as if they are interacting with each other and the third hovered for'about 15 minutes.' New Jersey Governor Phil Murphy said Monday night that the drones are'very sophisticated, explaining: 'The minute we get eyes on them [the drones], they go dark.' 'I don't blame people for being frustrated,' Gov Murphy continued, adding that he had spent most of Sunday coordinating on the issue with both the White House and the US Department of Homeland Security in the hope of getting answers. He said that the state received 49 sighting reports on Sunday night alone, with hundreds of locals sharing experiences on social media platforms. On Monday, Picatinny Arsenal, the Army facility in Morris County, confirmed it has had 11 sightings of'UFOs' over in its airspace in recent weeks.