Indian Ocean
Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning
Nath, Swaroop, Khadilkar, Harshad, Bhattacharyya, Pushpak
Query-focused Summarization (QfS) deals with systems that generate summaries from document(s) based on a query. Motivated by the insight that Reinforcement Learning (RL) provides a generalization to Supervised Learning (SL) for Natural Language Generation, and thereby performs better (empirically) than SL, we use an RL-based approach for this task of QfS. Additionally, we also resolve the conflict of employing RL in Transformers with Teacher Forcing. We develop multiple Policy Gradient networks, trained on various reward signals: ROUGE, BLEU, and Semantic Similarity, which lead to a 10-point improvement over the State-of-the-Art approach on the ROUGE-L metric for a benchmark dataset (ELI5). We also show performance of our approach in zero-shot setting for another benchmark dataset (DebatePedia) -- our approach leads to results comparable to baselines, which were specifically trained on DebatePedia. To aid the RL training, we propose a better semantic similarity reward, enabled by a novel Passage Embedding scheme developed using Cluster Hypothesis. Lastly, we contribute a gold-standard test dataset to further research in QfS and Long-form Question Answering (LfQA).
GlycoNMR: Dataset and benchmarks for NMR chemical shift prediction of carbohydrates with graph neural networks
Chen, Zizhang, Badman, Ryan Paul, Foley, Lachele, Woods, Robert, Hong, Pengyu
Molecular representation learning (MRL) is a powerful tool for bridging the gap between machine learning and chemical sciences, as it converts molecules into numerical representations while preserving their chemical features. These encoded representations serve as a foundation for various downstream biochemical studies, including property prediction and drug design. MRL has had great success with proteins and general biomolecule datasets. Yet, in the growing sub-field of glycoscience (the study of carbohydrates, where longer carbohydrates are also called glycans), MRL methods have been barely explored. This under-exploration can be primarily attributed to the limited availability of comprehensive and well-curated carbohydrate-specific datasets and a lack of Machine learning (ML) pipelines specifically tailored to meet the unique problems presented by carbohydrate data. Since interpreting and annotating carbohydrate-specific data is generally more complicated than protein data, domain experts are usually required to get involved. The existing MRL methods, predominately optimized for proteins and small biomolecules, also cannot be directly used in carbohydrate applications without special modifications. To address this challenge, accelerate progress in glycoscience, and enrich the data resources of the MRL community, we introduce GlycoNMR. GlycoNMR contains two laboriously curated datasets with 2,609 carbohydrate structures and 211,543 annotated nuclear magnetic resonance (NMR) chemical shifts for precise atomic-level prediction. We tailored carbohydrate-specific features and adapted existing MRL models to tackle this problem effectively. For illustration, we benchmark four modified MRL models on our new datasets.
Optimal Clustering of Discrete Mixtures: Binomial, Poisson, Block Models, and Multi-layer Networks
Lyu, Zhongyuan, Li, Ting, Xia, Dong
In this paper, we first study the fundamental limit of clustering networks when a multi-layer network is present. Under the mixture multi-layer stochastic block model (MMSBM), we show that the minimax optimal network clustering error rate, which takes an exponential form and is characterized by the Renyi divergence between the edge probability distributions of the component networks. We propose a novel two-stage network clustering method including a tensor-based initialization algorithm involving both node and sample splitting and a refinement procedure by likelihood-based Lloyd algorithm. Network clustering must be accompanied by node community detection. Our proposed algorithm achieves the minimax optimal network clustering error rate and allows extreme network sparsity under MMSBM. Numerical simulations and real data experiments both validate that our method outperforms existing methods. Oftentimes, the edges of networks carry count-type weights. We then extend our methodology and analysis framework to study the minimax optimal clustering error rate for mixture of discrete distributions including Binomial, Poisson, and multi-layer Poisson networks. The minimax optimal clustering error rates in these discrete mixtures all take the same exponential form characterized by the Renyi divergences. These optimal clustering error rates in discrete mixtures can also be achieved by our proposed two-stage clustering algorithm.
Israeli-owned ship targeted in suspected drone attack: Reports
A suspected drone attack has hit a container ship owned by an Israeli businessman in the Indian Ocean, according to a United States defence official. The attack was likely carried out using an Iranian-made Shahed-136 drone on Friday, an unnamed US defence official told The Associated Press news agency on Saturday. Pan-Arab satellite channel Al Mayadeen also reported that an Israeli ship had been targeted in the Indian Ocean. The drone targeted the Malta-flagged, French-operated CMA CGM Symi vessel while in international waters. The ship reportedly suffered damage after the drone exploded, but no crew members were injured.
US warship cruising Red Sea shoots down attack drones fired from Yemen
A US warship cruising the Red Sea has shot down drones fired from Houthi-held territory in Yemen, according to the US Central Command. The USS Thomas Hudner, a guided-missile destroyer, shot down "multiple one-way attack drones" launched on Thursday morning from Yemen's Houthi-controlled areas, CENTCOM said in a post on X, formerly Twitter. CENTCOM said there was no damage to the US vessel or injuries to its crew. On the morning (Yemen time) of November 23, the USS Thomas Hudner (DDG 116) shot down multiple one-way attack drones launched from Houthi controlled areas in Yemen. The drones were shot down while the U.S. warship was on patrol in the Red Sea.
US Navy destroyer shoots down drone from Yemen in the Red Sea
The U.S. Department of Defense released video footage of a U.S. air strike on a training and weapons facility in Abul Kamal, Syria. The USS Thomas Hudner, an Arleigh Burke-class destroyer, shot down a drone from Yemen in the Red Sea on Wednesday, two U.S. defense officials confirmed to Fox News. A defense official said the drone was shot down in self-defense. "The drone was heading towards the Hudner," the official said. The drone attack is the latest in a series of attacks on American troops stationed in the Middle East amid the ongoing Israel-Hamas war.
Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs
Chen, Jiefeng, Yoon, Jinsung, Ebrahimi, Sayna, Arik, Sercan O, Pfister, Tomas, Jha, Somesh
Large language models (LLMs) have recently shown great advances in a variety of tasks, including natural language understanding and generation. However, their use in high-stakes decision-making scenarios is still limited due to the potential for errors. Selective prediction is a technique that can be used to improve the reliability of the LLMs by allowing them to abstain from making predictions when they are unsure of the answer. In this work, we propose a novel framework for adaptation with self-evaluation to improve the selective prediction performance of LLMs. Our framework is based on the idea of using parameter-efficient tuning to adapt the LLM to the specific task at hand while improving its ability to perform self-evaluation. We evaluate our method on a variety of question-answering (QA) datasets and show that it outperforms state-of-the-art selective prediction methods. For example, on the CoQA benchmark, our method improves the AUACC from 91.23% to 92.63% and improves the AUROC from 74.61% to 80.25%.
U.S. Strikes Iran-Linked Facility in Syria in Round of Retaliation
For the second time in nearly two weeks, the United States carried out airstrikes against a facility used by Iran's Islamic Revolutionary Guards Corps and its proxies in eastern Syria early Thursday, ratcheting up retaliation for a steady stream of rocket and drone attacks against American forces in Iraq and Syria. The strikes by two Air Force F-15E jets against a weapons warehouse in Deir al Zour Province, Syria, came after U.S. airstrikes on Oct. 27 against similar targets in eastern Syria failed to deter Iran or its proxies in Syria and Iraq, which the Biden administration has blamed for the attacks. Not only have the attacks continued -- there have been at least 22 more since the American retaliatory strikes last month -- but Pentagon officials said they have become more dangerous. Iran-backed militias have packed even larger loads of explosives -- more than 80 pounds -- onto drones launched at American bases, U.S. officials said. "This precision self-defense strike is a response to a series of attacks against U.S. personnel in Iraq and Syria by I.R.G.C.-Quds Force affiliates," Defense Secretary Lloyd J. Austin III said in a statement.
Houthi Rebels Shot Down a U.S. Drone Off Yemen's Coast, Pentagon Says
A U.S. military surveillance drone was shot down off the coast of Yemen on Wednesday by Iran-backed Houthi rebels, the Pentagon said. Pentagon officials, speaking on the condition of anonymity to discuss operational matters, confirmed that the drone, an MQ-9 Reaper, had been shot down. But they would not say if the aircraft was armed, where it was flying from or other details. The downing of a Reaper drone, the mainstay of the American military's aerial surveillance fleet, was the latest escalation of violence between the United States and Iran-backed groups in Yemen, Iraq and Syria. The episodes have underscored the risks that the conflict between Israel and the Palestinian group Hamas could spiral into a wider war.
M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models
Zhang, Wenxuan, Aljunied, Sharifah Mahani, Gao, Chang, Chia, Yew Ken, Bing, Lidong
Despite the existence of various benchmarks for evaluating natural language processing models, we argue that human exams are a more suitable means of evaluating general intelligence for large language models (LLMs), as they inherently demand a much wider range of abilities such as language understanding, domain knowledge, and problem-solving skills. To this end, we introduce M3Exam, a novel benchmark sourced from real and official human exam questions for evaluating LLMs in a multilingual, multimodal, and multilevel context. M3Exam exhibits three unique characteristics: (1) multilingualism, encompassing questions from multiple countries that require strong multilingual proficiency and cultural knowledge; (2) multimodality, accounting for the multimodal nature of many exam questions to test the model's multimodal understanding capability; and (3) multilevel structure, featuring exams from three critical educational periods to comprehensively assess a model's proficiency at different levels. In total, M3Exam contains 12,317 questions in 9 diverse languages with three educational levels, where about 23\% of the questions require processing images for successful solving. We assess the performance of top-performing LLMs on M3Exam and find that current models, including GPT-4, still struggle with multilingual text, particularly in low-resource and non-Latin script languages. Multimodal LLMs also perform poorly with complex multimodal questions. We believe that M3Exam can be a valuable resource for comprehensively evaluating LLMs by examining their multilingual and multimodal abilities and tracking their development. Data and evaluation code is available at \url{https://github.com/DAMO-NLP-SG/M3Exam}.