The `pre-train, prompt, predict' paradigm of large language models (LLMs) has achieved remarkable success in open-domain question answering (OD-QA). However, few works explore this paradigm in the scenario of multi-document question answering (MD-QA), a task demanding a thorough understanding of the logical associations among the contents and structures of different documents. To fill this crucial gap, we propose a Knowledge Graph Prompting (KGP) method to formulate the right context in prompting LLMs for MD-QA, which consists of a graph construction module and a graph traversal module. For graph construction, we create a knowledge graph (KG) over multiple documents with nodes symbolizing passages or document structures (e.g., pages/tables), and edges denoting the semantic/lexical similarity between passages or intra-document structural relations. For graph traversal, we design an LLM-based graph traversal agent that navigates across nodes and gathers supporting passages assisting LLMs in MD-QA. The constructed graph serves as the global ruler that regulates the transitional space among passages and reduces retrieval latency. Concurrently, the graph traversal agent acts as a local navigator that gathers pertinent context to progressively approach the question and guarantee retrieval quality. Extensive experiments underscore the efficacy of KGP for MD-QA, signifying the potential of leveraging graphs in enhancing the prompt design for LLMs. Our code: https://github.com/YuWVandy/KG-LLM-MDQA.

Recently proposed BERT-based evaluation metrics for text generation perform well on standard benchmarks but are vulnerable to adversarial attacks, e.g., relating to information correctness. We argue that this stems (in part) from the fact that they are models of semantic similarity. In contrast, we develop evaluation metrics based on Natural Language Inference (NLI), which we deem a more appropriate modeling. We design a preference-based adversarial attack framework and show that our NLI based metrics are much more robust to the attacks than the recent BERT-based metrics. On standard benchmarks, our NLI based metrics outperform existing summarization metrics, but perform below SOTA MT metrics. However, when combining existing metrics with our NLI metrics, we obtain both higher adversarial robustness (15%-30%) and higher quality metrics as measured on standard benchmarks (+5% to 30%).

A Japanese-owned commercial tanker was attacked by a drone launched from Iran in the Indian Ocean on Saturday, causing a fire but leaving no one injured, the U.S. Department of Defense said. The Liberian-flagged, Netherlands-operated chemical tanker was struck about 370 kilometers off India "by a one-way attack drone fired from Iran," the Pentagon said in a statement, adding the fire on board has been extinguished. The Associated Press reported the tanker had ties with Israel, citing a British maritime security firm. The tanker, which was carrying 20 Indians and one Vietnamese, had left Saudi Arabia on Tuesday and was heading to Mangalore, southwestern India, according to the U.S. Defense Department and the Indian Coast Guard. But the vessel decided to head to Mumbai for repairs and to have the damage assessed with the assistance of the Indian Coast Guard.

Former CENTCOM Spokesperson and retired U.S. Army Colonel Joe Buccino discusses Iran's involvement in Houthi attacks and the U.S.' approach to deterrence and response. Iran appears to have struck a ship off the Indian coast with an unmanned aerial vehicle, a U.S. official told Fox News on Saturday. It comes as Houthi militants targeted multiple cargo ships on Saturday, as the group fired two anti-ship ballistic missiles into international shipping lanes located in the Southern Red Sea, according to U.S. Central Command. No ships were impacted by the ballistic missiles, officials said. The USS Laboon shot down four unmanned aerial drones on Saturday which originated from areas that the Houthis control in Yemen.

We investigate the predictive power of different machine learning algorithms to nowcast Madagascar's gross domestic product (GDP). We trained popular regression models, including linear regularized regression (Ridge, Lasso, Elastic-net), dimensionality reduction model (principal component regression), k-nearest neighbors algorithm (k-NN regression), support vector regression (linear SVR), and tree-based ensemble models (Random forest and XGBoost regressions), on 10 Malagasy quarterly macroeconomic leading indicators over the period 2007Q1--2022Q4, and we used simple econometric models as a benchmark. We measured the nowcast accuracy of each model by calculating the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). Our findings reveal that the Ensemble Model, formed by aggregating individual predictions, consistently outperforms traditional econometric models. We conclude that machine learning models can deliver more accurate and timely nowcasts of Malagasy economic performance and provide policymakers with additional guidance for data-driven decision making.

The increasing capability of Large Language Models to act as human-like social agents raises two important questions in the area of opinion dynamics. First, whether these agents can generate effective arguments that could be injected into the online discourse to steer the public opinion. Second, whether artificial agents can interact with each other to reproduce dynamics of persuasion typical of human social systems, opening up opportunities for studying synthetic social systems as faithful proxies for opinion dynamics in human populations. To address these questions, we designed a synthetic persuasion dialogue scenario on the topic of climate change, where a 'convincer' agent generates a persuasive argument for a 'skeptic' agent, who subsequently assesses whether the argument changed its internal opinion state. Different types of arguments were generated to incorporate different linguistic dimensions underpinning psycho-linguistic theories of opinion change. We then asked human judges to evaluate the persuasiveness of machine-generated arguments. Arguments that included factual knowledge, markers of trust, expressions of support, and conveyed status were deemed most effective according to both humans and agents, with humans reporting a marked preference for knowledge-based arguments. Our experimental framework lays the groundwork for future in-silico studies of opinion dynamics, and our findings suggest that artificial agents have the potential of playing an important role in collective processes of opinion formation in online social media.

In this paper, we present our finding that prepending a Task-Agnostic Prefix Prompt (TAPP) to the input improves the instruction-following ability of various Large Language Models (LLMs) during inference. TAPP is different from canonical prompts for LLMs in that it is a fixed prompt prepended to the beginning of every input regardless of the target task for zero-shot generalization. We observe that both base LLMs (i.e. not fine-tuned to follow instructions) and instruction-tuned models benefit from TAPP, resulting in 34.58% and 12.26% improvement on average, respectively. This implies that the instruction-following ability of LLMs can be improved during inference time with a fixed prompt constructed with simple heuristics. We hypothesize that TAPP assists language models to better estimate the output distribution by focusing more on the instruction of the target task during inference. In other words, such ability does not seem to be sufficiently activated in not only base LLMs but also many instruction-fine-tuned LLMs. All experiments are reproducible from https://github.com/seonghyeonye/TAPP.

Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent work has shown that a zero-pass approach, where parameters are interpreted directly without a forward/backward pass is feasible for some Transformer parameters, and for two-layer attention networks. In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on. We derive a simple theoretical framework to support our arguments and provide ample evidence for its validity. First, an empirical analysis showing that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Second, we present two applications of our framework: (a) aligning the parameters of different models that share a vocabulary, and (b) constructing a classifier without training by ``translating'' the parameters of a fine-tuned classifier to parameters of a different model that was only pretrained. Overall, our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only.

Gen Z voters are growing increasingly concerned about President Biden's mental fitness and "retiree" lifestyle as his approval rating slips to an all-time low in a national poll ahead of the 2024 election. "I don't mean to rag on the president of the United States, but honestly, he's acting like a retiree," Kale Ogunbor, a Republican Gen Z voter, told "Fox & Friends First" Wednesday. "It's been reported that over 40% of Joe Biden's presidency has been spent on vacation. And I think a lot of Americans, including Gen Z, don't want a president who seems more like he's retired the next four years after 2024." The president stands at 34% approval in a Monmouth University poll released Monday, with 61% giving Biden a thumbs down on his job performance.

In September, Elon Musk's brain-implant company Neuralink announced the much-anticipated news that it would start recruiting volunteers for a clinical trial to test its device. Known as a brain-computer interface, or BCI, it collects electrical activity from neurons and interprets those signals into commands to control an external device. While Musk has said he ultimately wants to merge humans with artificial intelligence, Neuralink's initial aim is to enable paralyzed people to control a cursor or keyboard with just their thoughts. Rival efforts to connect people's brains to computers are also moving forward. This year, Neuralink competitor Synchron demonstrated the long-term safety of its implant in patients.