Calais
Detecting and Mitigating Hallucinations in Multilingual Summarisation
Qiu, Yifu, Ziser, Yftah, Korhonen, Anna, Ponti, Edoardo M., Cohen, Shay B.
Hallucinations pose a significant challenge to the reliability of neural models for abstractive summarisation. While automatically generated summaries may be fluent, they often lack faithfulness to the original document. This issue becomes even more pronounced in low-resource settings, such as cross-lingual transfer. With the existing faithful metrics focusing on English, even measuring the extent of this phenomenon in cross-lingual settings is hard. To address this, we first develop a novel metric, mFACT, evaluating the faithfulness of non-English summaries, leveraging translation-based transfer from multiple English faithfulness metrics. We then propose a simple but effective method to reduce hallucinations with a cross-lingual transfer, which weighs the loss of each training example by its faithfulness score. Through extensive experiments in multiple languages, we demonstrate that mFACT is the metric that is most suited to detect hallucinations. Moreover, we find that our proposed loss weighting method drastically increases both performance and faithfulness according to both automatic and human evaluation when compared to strong baselines for cross-lingual transfer such as MAD-X. Our code and dataset are available at https://github.com/yfqiu-nlp/mfact-summ.
Flying robo-taxis eyed for Bay Area commuters
French inventor Frank Zapata grabbed headlines around the world this summer when he flew his hoverboard across the English channel from Pas de Calais, France, to the famous white cliffs of Dover. But Bay Area commuters may soon do Zapata one better by skimming above San Francisco Bay on autonomous, single-passenger drones being developed by a Peninsula start-up company with ties to Google. The automated drones are electrically powered, capable of vertical takeoff and landing, and would fly 10 feet above the water at 20 mph along a pre-determined flight path not subject to passenger controls. The drones' rotors are able to shift from vertical to horizontal alignment for efficient forward movement after takeoff. The company behind all this, three-year-old Kitty Hawk Corp., has personal financial backing from Google founder Larry Page, now CEO of Google's parent, Alphabet, who has long been interested in autonomous forms of transportation.