This paper proposes a vision-in-the-loop simulation environment for deep monocular pose estimation of a UAV operating in an ocean environment. Recently, a deep neural network with a transformer architecture has been successfully trained to estimate the pose of a UAV relative to the flight deck of a research vessel, overcoming several limitations of GPS-based approaches. However, validating the deep pose estimation scheme in an actual ocean environment poses significant challenges due to the limited availability of research vessels and the associated operational costs. To address these issues, we present a photo-realistic 3D virtual environment leveraging recent advancements in Gaussian splatting, a novel technique that represents 3D scenes by modeling image pixels as Gaussian distributions in 3D space, creating a lightweight and high-quality visual model from multiple viewpoints. This approach enables the creation of a virtual environment integrating multiple real-world images collected in situ. The resulting simulation enables the indoor testing of flight maneuvers while verifying all aspects of flight software, hardware, and the deep monocular pose estimation scheme. This approach provides a cost-effective solution for testing and validating the autonomous flight of shipboard UAVs, specifically focusing on vision-based control and estimation algorithms.

We develop a hierarchical LLM-task-motion planning and replanning framework to efficiently ground an abstracted human command into tangible Autonomous Underwater Vehicle (AUV) control through enhanced representations of the world. We also incorporate a holistic replanner to provide real-world feedback with all planners for robust AUV operation. While there has been extensive research in bridging the gap between LLMs and robotic missions, they are unable to guarantee success of AUV applications in the vast and unknown ocean environment. To tackle specific challenges in marine robotics, we design a hierarchical planner to compose executable motion plans, which achieves planning efficiency and solution quality by decomposing long-horizon missions into sub-tasks. At the same time, real-time data stream is obtained by a replanner to address environmental uncertainties during plan execution. Experiments validate that our proposed framework delivers successful AUV performance of long-duration missions through natural language piloting.

This article introduces a five-tiered route planner for accessing multiple nodes with multiple autonomous underwater vehicles (AUVs) that enables efficient task completion in stochastic ocean environments. First, the pre-planning tier solves the single-AUV routing problem to find the optimal giant route (GR), estimates the number of required AUVs based on GR segmentation, and allocates nodes for each AUV to access. Second, the route planning tier plans individual routes for each AUV. During navigation, the path planning tier provides each AUV with physical paths between any two points, while the actuation tier is responsible for path tracking and obstacle avoidance. Finally, in the stochastic ocean environment, deviations from the initial plan may occur, thus, an auction-based coordination tier drives online task coordination among AUVs in a distributed manner. Simulation experiments are conducted in multiple different scenarios to test the performance of the proposed planner, and the promising results show that the proposed method reduces AUV usage by 7.5% compared with the existing methods. When using the same number of AUVs, the fleet equipped with the proposed planner achieves a 6.2% improvement in average task completion rate.

Assistant Professor Wim van Rees and his team have developed simulations of self-propelled undulatory swimmers to better understand how fish-like deformable fins could improve propulsion in underwater devices, seen here in a top-down view. MIT ocean and mechanical engineers are using advances in scientific computing to address the ocean's many challenges, and seize its opportunities. There are few environments as unforgiving as the ocean. Its unpredictable weather patterns and limitations in terms of communications have left large swaths of the ocean unexplored and shrouded in mystery. "The ocean is a fascinating environment with a number of current challenges like microplastics, algae blooms, coral bleaching, and rising temperatures," says Wim van Rees, the ABS Career Development Professor at MIT. "At the same time, the ocean holds countless opportunities -- from aquaculture to energy harvesting and exploring the many ocean creatures we haven't discovered yet."

There are few environments as unforgiving as the ocean. Its unpredictable weather patterns and limitations in terms of communications have left large swaths of the ocean unexplored and shrouded in mystery. "The ocean is a fascinating environment with a number of current challenges like microplastics, algae blooms, coral bleaching, and rising temperatures," says Wim van Rees, the ABS Career Development Professor at MIT. "At the same time, the ocean holds countless opportunities -- from aquaculture to energy harvesting and exploring the many ocean creatures we haven't discovered yet." Ocean engineers and mechanical engineers, like van Rees, are using advances in scientific computing to address the ocean's many challenges, and seize its opportunities. These researchers are developing technologies to better understand our oceans, and how both organisms and human-made vehicles can move within them, from the micro scale to the macro scale.

Google's parent company, Alphabet, wants to throw the power of its AI behind a mission to monitor sea life around the ocean. According to a blog post, a project called Tidal - part of Alphabet's'X' division that develops'moonshot' projects - is creating a computer vision system that uses AI to monitor thousands of fish. The goal, says the company, is to help glean understanding of how over-fishing and other human impacts of fish populations affect sea life across the globe. With that information, Alphabet hopes that people will not just understand what happens beneath the ocean's surface, but help fix the problems that plague ocean life and habitats. 'One of the biggest barriers to protecting the ocean -- and our future -- is that we don't know much about what's going on under the water.

Each year the melting of the Charles River serves as a harbinger for warmer weather. Shortly thereafter is the return of budding trees, longer days, and flip-flops. For students of class 2.680 (Unmanned Marine Vehicle Autonomy, Sensing and Communications), the newly thawed river means it's time to put months of hard work into practice. "In underwater marine robotics, there is a unique need for artificial intelligence -- it's crucial," says MIT Professor Henrik Schmidt, the course's co-instructor. "And that is what we focus on in this class."