Goto

Collaborating Authors

 Government


Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluations

arXiv.org Artificial Intelligence

Large Language Models (LLMs) show promise as a writing aid for professionals performing legal analyses. However, LLMs can often hallucinate in this setting, in ways difficult to recognize by non-professionals and existing text evaluation metrics. In this work, we pose the question: when can machine-generated legal analysis be evaluated as acceptable? We introduce the neutral notion of gaps, as opposed to hallucinations in a strict erroneous sense, to refer to the difference between human-written and machine-generated legal analysis. Gaps do not always equate to invalid generation. Working with legal experts, we consider the CLERC generation task proposed in Hou et al. (2024b), leading to a taxonomy, a fine-grained detector for predicting gap categories, and an annotated dataset for automatic evaluation. Our best detector achieves 67% F1 score and 80% precision on the test set. Employing this detector as an automated metric on legal analysis generated by SOTA LLMs, we find around 80% contain hallucinations of different kinds.


Exact Recovery Guarantees for Parameterized Non-linear System Identification Problem under Adversarial Attacks

arXiv.org Artificial Intelligence

In this work, we study the system identification problem for parameterized non-linear systems using basis functions under adversarial attacks. Motivated by the LASSO-type estimators, we analyze the exact recovery property of a non-smooth estimator, which is generated by solving an embedded $\ell_1$-loss minimization problem. First, we derive necessary and sufficient conditions for the well-specifiedness of the estimator and the uniqueness of global solutions to the underlying optimization problem. Next, we provide exact recovery guarantees for the estimator under two different scenarios of boundedness and Lipschitz continuity of the basis functions. The non-asymptotic exact recovery is guaranteed with high probability, even when there are more severely corrupted data than clean data. Finally, we numerically illustrate the validity of our theory. This is the first study on the sample complexity analysis of a non-smooth estimator for the non-linear system identification problem.


A Scalable Tabletop Satellite Automation Testbed:Design And Experiments

arXiv.org Artificial Intelligence

INTRODUCTION The explosive growth of space technologies that are furthering the information age continue to elicit transformative advances that can chiefly be attributed largely to the inexpensive and reusable commercial launch capabilities. They form a basis for the space technology of the future, which include on-orbit servicing, assembling and manufacturing. Next generation space robotics technologies that focus on particular mechanisms and machines that are suitable for operations in micro and nano gravity environments form an important aspect of research. The confluence of advanced manufacturing technologies, novel materials, electronics and advanced computing architectures forms a fertile precursor for this research. One of the most versatile set of space mechanisms that can be a potential enabler are free-flying systems. Advances in CubeSat technology have continued to further the technology maturation of the free-flyer concept. Ground robotic platforms that focus on CubeSat technology enable accessible and low-cost spacecraft motion emulation solutions that contrast industrial and defense standards. The Land, Air, and Space Robotics (LASR) Laboratory at Texas A&M University has made significant contributions in the areas of dynamics, control and simulation of a Mobile Robotic System for 6-DOF Motion Emulation. The laboratory is home to a variety of robotic platforms that emulate satellite motions. One key robotic testbed is the Holonomic Omnidirectional Motion Emulation Robot (HOMER) that not only replaces air bearing tables for robotic proximity operation testing, but offers realistic 6-DOF motions to test sensors and actuators. The LASR Laboratory is now investigating the use of free-flying spacecraft modules in several different on-orbit, servicing and manufacturing (OSAM) activities as part of the Space University Research Initiative (SURI) funding opportunity awarded by the Air Force Office of Scientific Research to the "Breaking the'Launch Once, Use Once' Paradigm" Team, which consists of Carnegie Mellon University (CMU), University of New Mexico (UNM), Texas A&M University (TAMU), and the Northrop Grumman Corporation (NGC).


Benchmarking LLMs in Political Content Text-Annotation: Proof-of-Concept with Toxicity and Incivility Data

arXiv.org Artificial Intelligence

This article benchmarked the ability of OpenAI's GPTs and a number of open-source LLMs to perform annotation tasks on political content. We used a novel protest event dataset comprising more than three million digital interactions and created a gold standard that includes ground-truth labels annotated by human coders about toxicity and incivility on social media. We included in our benchmark Google's Perspective algorithm, which, along with GPTs, was employed throughout their respective APIs while the open-source LLMs were deployed locally. The findings show that Perspective API using a laxer threshold, GPT-4o, and Nous Hermes 2 Mixtral outperform other LLM's zero-shot classification annotations. In addition, Nous Hermes 2 and Mistral OpenOrca, with a smaller number of parameters, are able to perform the task with high performance, being attractive options that could offer good trade-offs between performance, implementing costs and computing time. Ancillary findings using experiments setting different temperature levels show that although GPTs tend to show not only excellent computing time but also overall good levels of reliability, only open-source LLMs ensure full reproducibility in the annotation.


Federated Learning in Adversarial Environments: Testbed Design and Poisoning Resilience in Cybersecurity

arXiv.org Artificial Intelligence

This paper presents the design and implementation of a Federated Learning (FL) testbed, focusing on its application in cybersecurity and evaluating its resilience against poisoning attacks. Federated Learning allows multiple clients to collaboratively train a global model while keeping their data decentralized, addressing critical needs for data privacy and security, particularly in sensitive fields like cybersecurity. Our testbed, built using the Flower framework, facilitates experimentation with various FL frameworks, assessing their performance, scalability, and ease of integration. Through a case study on federated intrusion detection systems, we demonstrate the testbed's capabilities in detecting anomalies and securing critical infrastructure without exposing sensitive network data. Comprehensive poisoning tests, targeting both model and data integrity, evaluate the system's robustness under adversarial conditions. Our results show that while federated learning enhances data privacy and distributed learning, it remains vulnerable to poisoning attacks, which must be mitigated to ensure its reliability in real-world applications.


Simulaci\'on de la distribuci\'on de alimento en el cultivo de camar\'on

arXiv.org Artificial Intelligence

This document presents the experimentation of 4 cases of food distribution for shrimp farming. The distributions are based on the location of the automatic feeders. Three cases applied in reality and a fourth case where the food is irrigated on the crop simultaneously and uniformly. In a first stage, the simulation of the three distribution cases is successfully adjusted to reality, where the trend of the shrimp growth curve is correlated with the historical data curve. A second stage where you experiment in 16 configurations that are based on the amount of food, the density of biomass and the distribution of the food. The simulation adopts the concepts of genetic algorithms to improve the population and fuzzy logic as an agent evaluation technique for decision-making against the quality of physical-chemical parameters in the simulated environment. The results of these interactions reveal a reduction in the simulated total culture time from 22 weeks to 14 weeks.


Language Models Learn Metadata: Political Stance Detection Case Study

arXiv.org Artificial Intelligence

Stance detection is a crucial NLP task with numerous applications in social science, from analyzing online discussions to assessing political campaigns. This paper investigates the optimal way to incorporate metadata into a political stance detection task. We demonstrate that previous methods combining metadata with language-based data for political stance detection have not fully utilized the metadata information; our simple baseline, using only party membership information, surpasses the current state-of-the-art. We then show that prepending metadata (e.g., party and policy) to political speeches performs best, outperforming all baselines, indicating that complex metadata inclusion systems may not learn the task optimally.


DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation

arXiv.org Artificial Intelligence

Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corruption/ noise affecting inputs. Existing approaches in TTA continuously adapt the DNN, leading to excessive resource consumption and performance degradation due to accumulation of error stemming from lack of supervision. In this work, we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of some corruption types, each one associated with a sub-network state tailored to correctly classify inputs affected by that corruption. After deployment, DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by (i) estimating the latent representation of the ongoing corruption; (ii) selecting the sub-network whose associated corruption is the closest in the latent space to the ongoing corruption; and (iii) adapting DNN state, so that its representation matches the ongoing corruption. This way, DARDA is more resource efficient and can swiftly adapt to new distributions caused by different corruptions without requiring a large variety of input data. Through experiments with two popular mobile edge devices - Raspberry Pi and NVIDIA Jetson Nano - we show that DARDA reduces energy consumption and average cache memory footprint respectively by 1.74x and 2.64x with respect to the state of the art, while increasing the performance by 10.4%, 5.7% and 4.4% on CIFAR-10, CIFAR-100 and TinyImagenet.


Regulating AI Is Easier Than You Think

TIME - Tech

Artificial intelligence is poised to deliver tremendous benefits to society. But, as many have pointed out, it could also bring unprecedented new horrors. As a general-purpose technology, the same tools that will advance scientific discovery could also be used to develop cyber, chemical, or biological weapons. Governing AI will require widely sharing its benefits while keeping the most powerful AI out of the hands of bad actors. The good news is that there is already a template on how to do just that.


Security News This Week: A Creative Trick Makes ChatGPT Spit Out Bomb-Making Instructions

WIRED

After Apple's product launch event this week, WIRED did a deep dive on the company's new secure server environment, known as Private Cloud Compute, which attempts to replicate in the cloud the security and privacy of processing data locally on users' individual devices. The goal is to minimize possible exposure of data processed for Apple Intelligence, the company's new AI platform. In addition to hearing about PCC from Apple's senior vice president of software engineering, Craig Federighi, WIRED readers also received a first look at content generated by Apple Intelligence's "Image Playground" feature as part of crucial updates on the recent birthday of Federighi's dog Bailey. Turning to privacy protection of a very different kind in another new AI service, WIRED looked at how users of the social media platform X can keep their data from being slurped up by the "unhinged" generative AI tool from xAI known as Grok AI. And in other news about Apple products, researchers developed a technique for using eye tracking to discern passwords and PINs people typed using 3D Apple Vision Pro avatars--a sort of keylogger for mixed reality.