Goto

Collaborating Authors

 vector space representation


Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings

Fitz, Stephen

arXiv.org Artificial Intelligence

As Large Language Models are deployed within Artificial Intelligence systems, that are increasingly integrated with human society, it becomes more important than ever to study their internal structures. Higher level abilities of LLMs such as GPT-3.5 emerge in large part due to informative language representations they induce from raw text data during pre-training on trillions of words. These embeddings exist in vector spaces of several thousand dimensions, and their processing involves mapping between multiple vector spaces, with total number of parameters on the order of trillions. Furthermore, these language representations are induced by gradient optimization, resulting in a black box system that is hard to interpret. In this paper, we take a look at the topological structure of neuronal activity in the "brain" of Chat-GPT's foundation language model, and analyze it with respect to a metric representing the notion of fairness. We develop a novel approach to visualize GPT's moral dimensions. We first compute a fairness metric, inspired by social psychology literature, to identify factors that typically influence fairness assessments in humans, such as legitimacy, need, and responsibility. Subsequently, we summarize the manifold's shape using a lower-dimensional simplicial complex, whose topology is derived from this metric. We color it with a heat map associated with this fairness metric, producing human-readable visualizations of the high-dimensional sentence manifold. Our results show that sentence embeddings based on GPT-3.5 can be decomposed into two submanifolds corresponding to fair and unfair moral judgments. This indicates that GPT-based language models develop a moral dimension within their representation spaces and induce an understanding of fairness during their training process.


Senior Data Engineer

#artificialintelligence

As one of the largest North American automotive suppliers, Bosch develops Driver Assistance functions like Adaptive Cruise Control (ACC), Predictive Emergency Brake Systems (PEBS), Lane Departure Warning/Keeping Systems (LDW, LKS), Predictive Pedestrian Protection, Road Sign recognition, head light control, Advanced Parking Assistance, and many more. For these functions Bosch has all necessary sensors in our portfolio (e.g. In our offices in Plymouth, MI and Palo Alto, CA we develop state of the art systems as well as advanced features leading to partly/highly automated driving. Join us to become part of the exciting and growing field of Driver Assistance. We are on the mission to turn latest technology into outstanding Bosch products and services.


Senior ML Ops Engineer

#artificialintelligence

As one of the largest North American automotive suppliers, Bosch develops Driver Assistance functions like Adaptive Cruise Control (ACC), Predictive Emergency Brake Systems (PEBS), Lane Departure Warning/Keeping Systems (LDW, LKS), Predictive Pedestrian Protection, Road Sign recognition, head light control, Advanced Parking Assistance, and many more. For these functions Bosch has all necessary sensors in our portfolio (e.g. In our office in Plymouth, MI, we develop state of the art systems as well as advanced features leading to partly/highly automated driving. Join us to become part of the exciting and growing field of Driver Assistance. In our team we are developing the perception of the next generation of automatic parking systems.


Fake News Detection: a comparison between available Deep Learning techniques in vector space

Singh, Lovedeep

arXiv.org Artificial Intelligence

Fake News Detection is an essential problem in the field of Natural Language Processing. The benefits of an effective solution in this area are manifold for the goodwill of society. On a surface level, it broadly matches with the general problem of text classification. Researchers have proposed various approaches to tackle fake news using simple as well as some complex techniques. In this paper, we try to make a comparison between the present Deep Learning techniques by representing the news instances in some vector space using a combination of common mathematical operations with available vector space representations. We do a number of experiments using various combinations and permutations. Finally, we conclude with a sound analysis of the results and evaluate the reasons for such results.


Learning Conceptual Space Representations of Interrelated Concepts

Bouraoui, Zied, Schockaert, Steven

arXiv.org Artificial Intelligence

Several recently proposed methods aim to learn conceptual space representations from large text collections. These learned representations asso- ciate each object from a given domain of interest with a point in a high-dimensional Euclidean space, but they do not model the concepts from this do- main, and can thus not directly be used for catego- rization and related cognitive tasks. A natural solu- tion is to represent concepts as Gaussians, learned from the representations of their instances, but this can only be reliably done if sufficiently many in- stances are given, which is often not the case. In this paper, we introduce a Bayesian model which addresses this problem by constructing informative priors from background knowledge about how the concepts of interest are interrelated with each other. We show that this leads to substantially better pre- dictions in a knowledge base completion task.


Detecting Events and Patterns in Large-Scale User Generated Textual Streams with Statistical Learning Methods

Lampos, Vasileios

arXiv.org Machine Learning

A vast amount of textual web streams is influenced by events or phenomena emerging in the real world. The social web forms an excellent modern paradigm, where unstructured user generated content is published on a regular basis and in most occasions is freely distributed. The present Ph.D. Thesis deals with the problem of inferring information - or patterns in general - about events emerging in real life based on the contents of this textual stream. We show that it is possible to extract valuable information about social phenomena, such as an epidemic or even rainfall rates, by automatic analysis of the content published in Social Media, and in particular Twitter, using Statistical Machine Learning methods. An important intermediate task regards the formation and identification of features which characterise a target event; we select and use those textual features in several linear, non-linear and hybrid inference approaches achieving a significantly good performance in terms of the applied loss function. By examining further this rich data set, we also propose methods for extracting various types of mood signals revealing how affective norms - at least within the social web's population - evolve during the day and how significant events emerging in the real world are influencing them. Lastly, we present some preliminary findings showing several spatiotemporal characteristics of this textual information as well as the potential of using it to tackle tasks such as the prediction of voting intentions.