Goto

Collaborating Authors

 Government


FoundTS: Comprehensive and Unified Benchmarking of Foundation Models for Time Series Forecasting

arXiv.org Artificial Intelligence

Time Series Forecasting (TSF) is key functionality in numerous fields, including in finance, weather services, and energy management. While TSF methods are emerging these days, many of them require domain-specific data collection and model training and struggle with poor generalization performance on new domains. Foundation models aim to overcome this limitation. Pre-trained on large-scale language or time series data, they exhibit promising inferencing capabilities in new or unseen data. This has spurred a surge in new TSF foundation models. We propose a new benchmark, FoundTS, to enable thorough and fair evaluation and comparison of such models. FoundTS covers a variety of TSF foundation models, including those based on large language models and those pretrained on time series. Next, FoundTS supports different forecasting strategies, including zero-shot, few-shot, and full-shot, thereby facilitating more thorough evaluations. Finally, FoundTS offers a pipeline that standardizes evaluation processes such as dataset splitting, loading, normalization, and few-shot sampling, thereby facilitating fair evaluations. Building on this, we report on an extensive evaluation of TSF foundation models on a broad range of datasets from diverse domains and with different statistical characteristics. Specifically, we identify pros and cons and inherent limitations of existing foundation models, and we identify directions for future model design. We make our code and datasets available at https://anonymous.4open.science/r/FoundTS-C2B0.


Google's antitrust gut punch and the Trump wild card

MIT Technology Review

The case was first filed back in 2020. Then in August, Judge Amit Mehta ruled in favor of DOJ (and against Google), finding that Google ran its business as an illegal monopoly. Now, the DOJ has made its case for what it thinks Google should have to do in the wake of that verdict. Next, Google will propose its own set of remedies to the court. Finally, Judge Mehta will have to decide which, if any, of these remedies to enact. The government starts by calling for an end to "third party payments." This means Google would have to stop paying the likes of Apple and Mozilla to make Google search the default engine in those companies' browsers and devices.


3 things Dr. Oz can do as CMS administrator to help fix American health care

FOX News

Dr. Mehmet Oz, nominated by President-elect Trump to be the next CMS administrator, will have his hands full when he takes the reins of our agency that provides health insurance for over 140 million Americans and accounts for one quarter of the nation's budget. Through bold and principled leadership and effective communication, he holds the potential to make a positive, lasting difference. It's no secret our health care system is broken. We continue to spend more and more money without any clear return on investment when it comes to our nation's health and well-being. In fact, nearly 50% of Americans live with one chronic health condition, such as diabetes, hypertension, or obesity.


Meta now allows military agencies to access its AI software. It poses a moral dilemma for everybody who uses it

AIHub

Meta will make its generative artificial intelligence (AI) models available to the United States' government, the tech giant has announced, in a controversial move that raises a moral dilemma for everyone who uses the software. Meta last week revealed it would make the models, known as Llama, available to government agencies, "including those that are working on defence and national security applications, and private sector partners supporting their work". The decision appears to contravene Meta's own policy which lists a range of prohibited uses for Llama, including "[m]ilitary, warfare, nuclear industries or applications" as well as espionage, terrorism, human trafficking and exploitation or harm to children. Meta's exception also reportedly applies to similar national security agencies in the United Kingdom, Canada, Australia and New Zealand. It came just three days after Reuters revealed China has reworked Llama for its own military purposes.


Russia-Ukraine war: List of key events, day 1,005

Al Jazeera

Russia's air defence systems destroyed seven Ukrainian missiles overnight over the Kursk region, Kursk regional governor Alexei Smirnov wrote on his Telegram channel. Falling debris from destroyed Ukrainian drones sparked a fire at an industrial facility in Russia's Kaluga, according to regional governor Vladislav Shapsha. He said there were no injuries and that three drones were destroyed. Russian forces captured a British mercenary, who identified himself as James Scott Rhys, who is fighting with the Ukrainian army in Russia's Kursk region, a security source told Russia's RIA Novosti state news agency. Air defences were in operation in Kyiv in response to a new Russian drone attack, Mayor Vitali Klitschko said on Telegram.


Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete?

arXiv.org Artificial Intelligence

Algorithmic agents are used in a variety of competitive decision settings, notably in making pricing decisions in contexts that range from online retail to residential home rentals. Business managers, algorithm designers, legal scholars, and regulators alike are all starting to consider the ramifications of "algorithmic collusion." We study the emergent behavior of multi-armed bandit machine learning algorithms used in situations where agents are competing, but they have no information about the strategic interaction they are engaged in. Using a general-form repeated Prisoner's Dilemma game, agents engage in online learning with no prior model of game structure and no knowledge of competitors' states or actions (e.g., no observation of competing prices). We show that these context-free bandits, with no knowledge of opponents' choices or outcomes, still will consistently learn collusive behavior - what we call "naive collusion." We primarily study this system through an analytical model and examine perturbations to the model through simulations. Our findings have several notable implications for regulators. First, calls to limit algorithms from conditioning on competitors' prices are insufficient to prevent algorithmic collusion. This is a direct result of collusion arising even in the naive setting. Second, symmetry in algorithms can increase collusion potential. This highlights a new, simple mechanism for "hub-and-spoke" algorithmic collusion. A central distributor need not imbue its algorithm with supra-competitive tendencies for apparent collusion to arise; it can simply arise by using certain (common) machine learning algorithms. Finally, we highlight that collusive outcomes depend starkly on the specific algorithm being used, and we highlight market and algorithmic conditions under which it will be unknown a priori whether collusion occurs.


What can LLM tell us about cities?

arXiv.org Artificial Intelligence

This study explores the capabilities of large language models (LLMs) in providing knowledge about cities and regions on a global scale. We employ two methods: directly querying the LLM for target variable values and extracting explicit and implicit features from the LLM correlated with the target variable. Our experiments reveal that LLMs embed a broad but varying degree of knowledge across global cities, with ML models trained on LLM-derived features consistently leading to improved predictive accuracy. Additionally, we observe that LLMs demonstrate a certain level of knowledge across global cities on all continents, but it is evident when they lack knowledge, as they tend to generate generic or random outputs for unfamiliar tasks. These findings suggest that LLMs can offer new opportunities for data-driven decision-making in the study of cities.


Teaching Smaller Language Models To Generalise To Unseen Compositional Questions (Full Thesis)

arXiv.org Artificial Intelligence

Pretrained large Language Models (LLMs) are able to answer questions that are unlikely to have been encountered during training. However a diversity of potential applications exist in the broad domain of reasoning systems and considerations such as latency, cost, available compute resource and internet connectivity are relevant in determining an appropriate approach. We consider the setting where some local compute capacity is available at inference time but internet connectivity is not. Similar to a general-purpose LLM, we assume that our much smaller Reasoning Models may be asked arbitrary questions from unknown distributions, so we focus on evaluation in an unseen setting. We train our models to answer diverse questions by instilling an ability to reason over a retrieved context. We acquire context from two knowledge sources; a Wikipedia corpus queried using a multi-hop dense retrieval system with novel extensions, and from rationales generated from a larger Language Model optimised to run in a lower resource environment. Our main contributions: We propose novel methods to show that our model is capable of answering contextualised questions without memorisation. We establish a comprehensive set of baseline results on unseen evaluation datasets. We show that the addition of novel retrieval-augmented training datasets (RATD) to the training regime of the Reasoning Model significantly improves results. We demonstrate further significant improvement through the application of methods for combining knowledge from two sources. The first method (RR) involves training a novel Rationale Ranking model to score both generated rationales and retrieved contexts with respect to relevance and truthfulness. We use the scores to derive combined contexts. We also show that utilising the RATD datasets enables our model to become proficient at utilising combined noisy contexts.


A Graph Neural Architecture Search Approach for Identifying Bots in Social Media

arXiv.org Artificial Intelligence

Social media platforms, including X, Facebook, and Instagram, host millions of daily users, giving rise to bots-automated programs disseminating misinformation and ideologies with tangible real-world consequences. While bot detection in platform X has been the area of many deep learning models with adequate results, most approaches neglect the graph structure of social media relationships and often rely on hand-engineered architectures. Our work introduces the implementation of a Neural Architecture Search (NAS) technique, namely Deep and Flexible Graph Neural Architecture Search (DFG-NAS), tailored to Relational Graph Convolutional Neural Networks (RGCNs) in the task of bot detection in platform X. Our model constructs a graph that incorporates both the user relationships and their metadata. Then, DFG-NAS is adapted to automatically search for the optimal configuration of Propagation and Transformation functions in the RGCNs. Our experiments are conducted on the TwiBot-20 dataset, constructing a graph with 229,580 nodes and 227,979 edges. We study the five architectures with the highest performance during the search and achieve an accuracy of 85.7%, surpassing state-of-the-art models. Our approach not only addresses the bot detection challenge but also advocates for the broader implementation of NAS models in neural network design automation.


Towards Agentic Schema Refinement

arXiv.org Artificial Intelligence

Understanding the meaning of data is crucial for performing data analysis, yet for the users to gain insight into the content and structure of their database, a tedious data exploration process is often required [2, 16]. A common industry practice taken on by specialists such as Knowledge Engineers is to explicitly construct an intermediate layer between the database and the user -- a semantic layer -- abstracting away certain details of the database schema in favor of clearer data semantics [3, 10]. In the era of Large Language Models (LLMs), industry practitioners and researchers attempt to circumvent this costly process using LLM-powered Natural Language Interfaces [4, 6, 12, 18, 19, 22]. The promise of such Text-to-SQL solutions is to allow users without technical expertise to seamlessly interact with databases. For example, a new company employee could effectively issue queries in natural language without programming expertise or even explicit knowledge of the database structure, e.g., knowing the names of entities or properties, the exact location of data sources, etc.