Trending Now Say More Look Up In My Bag Mashable's Best: E-readers, robovacs, laptops, earbuds, smart home and more Mashable @Home Mashable Selects Top creators, ranked Gift Ideas For Everyone On Your List Creator Playbook AI at School Safety Net All Series A new mission to find Amelia Earhart's plane turns to AI for help AI can map the ocean floor in ways we never imagined -- and possibly uncover millions of shipwrecks and downed planes. For almost 90 years, the mystery of what happened to Amelia Earhart has endured. She and her plane disappeared somewhere over the central Pacific Ocean on July 2, 1937. Since then, it seems like every few years, an expedition sets out to find her plane. The boats and recovery teams have always been armed with the latest equipment, historical diaries, and maps.

Evaluating machine-generated text remains a significant challenge in NLP, especially for non-English languages. Current methodologies, including automated metrics, human assessments, and LLM-based evaluations, predominantly focus on English, revealing a significant gap in multilingual evaluation frameworks. We introduce the Cross Lingual Auto Evaluation (CIA) Suite, an extensible framework that includes evaluator LLMs (Hercule) and a novel test set (Recon) specifically designed for multilingual evaluation. Our test set features 500 human-annotated instructions spanning various task capabilities along with human judgment scores across six languages. This would enable benchmarking of general-purpose multilingual LLMs and facilitate meta-evaluation of Evaluator LLMs. The proposed model, Hercule, is a cross-lingual evaluation model that addresses the scarcity of reference answers in the target language by learning to assign scores to responses based on easily available reference answers in English. Our experiments demonstrate that Hercule aligns more closely with human judgments compared to proprietary models, demonstrating the effectiveness of such cross-lingual evaluation in low resource scenarios. Further, it is also effective in zero-shot evaluation on unseen languages. This study is the first comprehensive examination of cross-lingual evaluation using LLMs, presenting a scalable and effective approach for multilingual assessment. All code, datasets, and models will be publicly available to enable further research in this important area.

A robotic ship from the University of New Hampshire's Marine School that can map the ocean floor is part of the latest effort to find out what happened to famed pilot Amelia Earhart, who disappeared over the Pacific Ocean eight decades ago. The autonomous vessel, known as BEN, the Bathymetric Explorer and Navigator, will be mapping the seafloor near the island where Earhart sent her last radio transmission. The area is too deep for divers and too shallow for safe navigation by deep-water sonar systems. Maps produced by BEN will be used to target later dives by remotely operated vehicles, searching for remnants of Earhart's plane. The work is part of the mission led by oceanographer Robert Ballard, best known for finding the wreck of the Titanic, to look into the disappearance of Earhart in 1937.