Goto

Collaborating Authors

 Drones


ES-PTAM: Event-based Stereo Parallel Tracking and Mapping

arXiv.org Artificial Intelligence

Visual Odometry (VO) and SLAM are fundamental components for spatial perception in mobile robots. Despite enormous progress in the field, current VO/SLAM systems are limited by their sensors' capability. Event cameras are novel visual sensors that offer advantages to overcome the limitations of standard cameras, enabling robots to expand their operating range to challenging scenarios, such as high-speed motion and high dynamic range illumination. We propose a novel event-based stereo VO system by combining two ideas: a correspondence-free mapping module that estimates depth by maximizing ray density fusion and a tracking module that estimates camera poses by maximizing edge-map alignment. We evaluate the system comprehensively on five real-world datasets, spanning a variety of camera types (manufacturers and spatial resolutions) and scenarios (driving, flying drone, hand-held, egocentric, etc). The quantitative and qualitative results demonstrate that our method outperforms the state of the art in majority of the test sequences by a margin, e.g., trajectory error reduction of 45% on RPG dataset, 61% on DSEC dataset, and 21% on TUM-VIE dataset. To benefit the community and foster research on event-based perception systems, we release the source code and results: https://github.com/tub-rip/ES-PTAM


Aftermath of Russian missile and drone attacks on Ukraine

Al Jazeera

Russia has launched a second straight day of missile and drone attacks, targeting several Ukrainian regions and killing at least five people. Two people were killed when a hotel was "wiped out" by a missile in the central city of Kryvyi Rih, regional officials said on Tuesday. Three died in drone attacks on the southeastern city of Zaporizhzhia. Ukrainian President Volodymyr Zelenskyy on Tuesday said Kyiv would retaliate and asked allies to consider joint air defence operations and provide long-range weapons capabilities. Ukraine's air force said it downed five of 10 incoming Russian missiles and 60 of 81 drones in Tuesday's attacks.


'Putin is vindictive': Russia pounds Ukraine as Kyiv pursues Kursk assault

Al Jazeera

Kyiv, Ukraine โ€“ Russia's aerial attack on Ukraine was colossal. Moving in waves from several directions and at different speeds and heights, 127 missiles and 109 drones attacked 15 of Ukraine's 24 regions. The attack is being seen in Ukraine as Russian President Vladimir Putin's revenge for Kyiv's daring incursion into the western Russian region of Kursk that began in early August and has resulted in the apparent takeover of more than 1,000sq kilometres (386sq miles). "He is a vindictive person, he got offended," General Lieutenant Ihor Romanenko, ex-deputy head of the General Staff of Ukraine's Armed Forces, told Al Jazeera. The attack began in predawn darkness on Monday as buzzing swarms of explosives-laden heavy drones took off from the Azov Sea town of Yeisk in southwestern Russia.


Russia launches another wave of missiles and drones at Ukraine

Al Jazeera

Russia has launched waves of missile and drone attacks targeting several Ukrainian regions killing at least four people, according to Ukraine's military, a day after it carried out a "massive" attack on Ukraine's power grid. Two people were killed when a hotel was "wiped out" in the central Ukrainian city of Kryvyi Rih, regional officials said on Tuesday. Two more people died in drone attacks on the city of Zaporizhzhia, east of Kryvyi Rih. The Kyiv region's air defence systems were deployed several times overnight to repel missiles and drones targeting Kyiv, the regional military administration said on the Telegram messaging app. Ukrainian air defences shot down about 15 drones and several missiles near the Ukrainian capital during Russia's overnight attack, Serhiy Popko, head of Kyiv's military administration, said on Tuesday morning.


SpecGuard: Specification Aware Recovery for Robotic Autonomous Vehicles from Physical Attacks

arXiv.org Artificial Intelligence

Robotic Autonomous Vehicles (RAVs) rely on their sensors for perception, and follow strict mission specifications (e.g., altitude, speed, and geofence constraints) for safe and timely operations. Physical attacks can corrupt the RAVs' sensors, resulting in mission failures. Recovering RAVs from such attacks demands robust control techniques that maintain compliance with mission specifications even under attacks to ensure the RAV's safety and timely operations. We propose SpecGuard, a technique that complies with mission specifications and performs safe recovery of RAVs. There are two innovations in SpecGuard. First, it introduces an approach to incorporate mission specifications and learn a recovery control policy using Deep Reinforcement Learning (Deep-RL). We design a compliance-based reward structure that reflects the RAV's complex dynamics and enables SpecGuard to satisfy multiple mission specifications simultaneously. Second, SpecGuard incorporates state reconstruction, a technique that minimizes attack induced sensor perturbations. This reconstruction enables effective adversarial training, and optimizing the recovery control policy for robustness under attacks. We evaluate SpecGuard in both virtual and real RAVs, and find that it achieves 92% recovery success rate under attacks on different sensors, without any crashes or stalls. SpecGuard achieves 2X higher recovery success than prior work, and incurs about 15% performance overhead on real RAVs.


AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models

arXiv.org Artificial Intelligence

Aerospace embodied intelligence aims to empower unmanned aerial vehicles (UAVs) and other aerospace platforms to achieve autonomous perception, cognition, and action, as well as egocentric active interaction with humans and the environment. The aerospace embodied world model serves as an effective means to realize the autonomous intelligence of UAVs and represents a necessary pathway toward aerospace embodied intelligence. However, existing embodied world models primarily focus on ground-level intelligent agents in indoor scenarios, while research on UAV intelligent agents remains unexplored. To address this gap, we construct the first large-scale real-world image-text pre-training dataset, AerialAgent-Ego10k, featuring urban drones from a first-person perspective. We also create a virtual image-text-pose alignment dataset, CyberAgent Ego500k, to facilitate the pre-training of the aerospace embodied world model. For the first time, we clearly define 5 downstream tasks, i.e., aerospace embodied scene awareness, spatial reasoning, navigational exploration, task planning, and motion decision, and construct corresponding instruction datasets, i.e., SkyAgent-Scene3k, SkyAgent-Reason3k, SkyAgent-Nav3k and SkyAgent-Plan3k, and SkyAgent-Act3k, for fine-tuning the aerospace embodiment world model. Simultaneously, we develop SkyAgentEval, the downstream task evaluation metrics based on GPT-4, to comprehensively, flexibly, and objectively assess the results, revealing the potential and limitations of 2D/3D visual language models in UAV-agent tasks. Furthermore, we integrate over 10 2D/3D visual-language models, 2 pre-training datasets, 5 finetuning datasets, more than 10 evaluation metrics, and a simulator into the benchmark suite, i.e., AeroVerse, which will be released to the community to promote exploration and development of aerospace embodied intelligence.


Distributed Planning for Rigid Robot Formations with Probabilistic Collision Avoidance

arXiv.org Artificial Intelligence

In [11], an APF is applied both is often required for tasks such as inspection and reconnaissance to the VRB and the robots, where the potential acting on [1]. The goal of formation planning algorithms is to the VRB is calculated at a central node and the potentials compute movements of the robots such that a formation is acting on the robots are calculated locally on each robot. In maintained while they perform their respective tasks and avoid [12], convex optimisation is used to find the optimal similarity collisions with obstacles and each other. Formation planning transformation of a configuration that minimises the distance algorithms can be categorised as centralised or distributed. In travelled by the robots, with constraints on the velocities of the centralised methods, all information is gathered at a central robots and environmental constraints on the formation. In [13], location where the plans are computed, while in distributed [14], a method for navigating a formation toward a desired methods all robots participate in the computation of the plans state, by iteratively computing the optimal VRB within the and coordinate via communication, requiring inter-robot coordination largest convex polytope containing the current formation, is mechanisms [1]. Centralised methods can be simpler presented. In [15], the authors demonstrate that finding the to implement since they do not require inter-robot coordination optimal rotation, translation, and assignment of the robots mechanisms, but have the drawback of system dependency in a VRB can be solved separately. In [16], consensus is on a single computer representing a single point of failure.


Multi-weather Cross-view Geo-localization Using Denoising Diffusion Models

arXiv.org Artificial Intelligence

Cross-view geo-localization in GNSS-denied environments aims to determine an unknown location by matching drone-view images with the correct geo-tagged satellite-view images from a large gallery. Recent research shows that learning discriminative image representations under specific weather conditions can significantly enhance performance. However, the frequent occurrence of unseen extreme weather conditions hinders progress. This paper introduces MCGF, a Multi-weather Cross-view Geo-localization Framework designed to dynamically adapt to unseen weather conditions. MCGF establishes a joint optimization between image restoration and geo-localization using denoising diffusion models. For image restoration, MCGF incorporates a shared encoder and a lightweight restoration module to help the backbone eliminate weather-specific information. For geo-localization, MCGF uses EVA-02 as a backbone for feature extraction, with cross-entropy loss for training and cosine distance for testing. Extensive experiments on University160k-WX demonstrate that MCGF achieves competitive results for geo-localization in varying weather conditions.


U2UData: A Large-scale Cooperative Perception Dataset for Swarm UAVs Autonomous Flight

arXiv.org Artificial Intelligence

Modern perception systems for autonomous flight are sensitive to occlusion and have limited long-range capability, which is a key bottleneck in improving low-altitude economic task performance. Recent research has shown that the UAV-to-UAV (U2U) cooperative perception system has great potential to revolutionize the autonomous flight industry. However, the lack of a large-scale dataset is hindering progress in this area. This paper presents U2UData, the first large-scale cooperative perception dataset for swarm UAVs autonomous flight. The dataset was collected by three UAVs flying autonomously in the U2USim, covering a 9 km$^2$ flight area. It comprises 315K LiDAR frames, 945K RGB and depth frames, and 2.41M annotated 3D bounding boxes for 3 classes. It also includes brightness, temperature, humidity, smoke, and airflow values covering all flight routes. U2USim is the first real-world mapping swarm UAVs simulation environment. It takes Yunnan Province as the prototype and includes 4 terrains, 7 weather conditions, and 8 sensor types. U2UData introduces two perception tasks: cooperative 3D object detection and cooperative 3D object tracking. This paper provides comprehensive benchmarks of recent cooperative perception algorithms on these tasks.


Russia, Ukraine trade drone attacks in renewed escalation

Al Jazeera

Russia has launched several strikes across Ukraine, killing at least five people and wounding several, in an attack that appeared to target energy infrastructure. Ukraine also launched a drone attack on Russia's central region of Saratov, injuring four. The exchange began around midnight on Sunday and continued beyond daybreak on Monday. Ukraine's air force reported multiple groups of Russian drones moving towards its eastern, northern, southern, and central regions, followed by numerous cruise and ballistic missiles. Authorities in at least six Ukrainian regions said blasts had been heard.