distinguishing
Distinguishing the Knowable from the Unknowable with Language Models
Ahdritz, Gustaf, Qin, Tian, Vyas, Nikhil, Barak, Boaz, Edelman, Benjamin L.
We study the feasibility of identifying epistemic uncertainty (reflecting a lack of knowledge), as opposed to aleatoric uncertainty (reflecting entropy in the underlying distribution), in the outputs of large language models (LLMs) over free-form text. In the absence of ground-truth probabilities, we explore a setting where, in order to (approximately) disentangle a given LLM's uncertainty, a significantly larger model stands in as a proxy for the ground truth. We show that small linear probes trained on the embeddings of frozen, pretrained models accurately predict when larger models will be more confident at the token level and that probes trained on one text domain generalize to others. Going further, we propose a fully unsupervised method that achieves non-trivial accuracy on the same task. Taken together, we interpret these results as evidence that LLMs naturally contain internal representations of different types of uncertainty that could potentially be leveraged to devise more informative indicators of model confidence in diverse practical settings.
- North America > United States > Louisiana > Orleans Parish > New Orleans (0.04)
- Europe > United Kingdom > Scotland (0.04)
- Europe > Sweden > Stockholm > Stockholm (0.04)
- (14 more...)
- Personal (0.92)
- Research Report (0.82)
- Education (0.93)
- Leisure & Entertainment > Sports > Tennis (0.92)
- Law > Criminal Law (0.67)
- (2 more...)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Performance Analysis > Accuracy (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
Distinguishing between chatbots and conversational AI
Chatbots operate based on the limited and predetermined flow that can activate a psychotherapist's conversation using a script. Chatbots usually carry out conversations by deploying a specific pattern that gives users an illusion of understanding on the part of the program. However, chatbots do not have an inbuilt framework for contextualizing events. Chatbots entail communication between humans and machines. In this case, humans tend to believe they are conversing with a fellow human.
Single Test Image-Based Automated Machine Learning System for Distinguishing between Trait and Diseased Blood Samples
Nasser, Sahar A., Paul, Debjani, Awate, Suyash P.
We introduce a machine learning-based method for fully automated diagnosis of sickle cell disease of poor-quality unstained images of a mobile microscope. Our method is capable of distinguishing between diseased, trait (carrier), and normal samples unlike the previous methods that are limited to distinguishing the normal from the abnormal samples only. The novelty of this method comes from distinguishing the trait and the diseased samples from challenging images that have been captured directly in the field. The proposed approach contains two parts, the segmentation part followed by the classification part. We use a random forest algorithm to segment such challenging images acquitted through a mobile phone-based microscope. Then, we train two classifiers based on a random forest (RF) and a support vector machine (SVM) for classification. The results show superior performances of both of the classifiers not only for images which have been captured in the lab, but also for the ones which have been acquired in the field itself.
- Asia > India (0.15)
- North America > United States > New York (0.04)
Distinguishing between Narrow AI, General AI and Super AI
The age of AI is upon us; in many ways, it's engulfing us. We're overwhelmed with information, articles and opinions on AI. Experts and non-experts alike are attempting to envision a future driven by the rise of this exponential technology. Because of the constant flow of information on AI, it's becoming increasingly difficult to pinpoint what exactly AI is. Few of us are able to actually define artificial intelligence.
- Leisure & Entertainment (0.48)
- Health & Medicine (0.31)
Distinguishing Between Roles of Football Players in Play-by-play Match Event Data
Aalbers, Bart, Van Haaren, Jan
Over the last few decades, the player recruitment process in professional football has evolved into a multi-billion industry and has thus become of vital importance. To gain insights into the general level of their candidate reinforcements, many professional football clubs have access to extensive video footage and advanced statistics. However, the question whether a given player would fit the team's playing style often still remains unanswered. In this paper, we aim to bridge that gap by proposing a set of 21 player roles and introducing a method for automatically identifying the most applicable roles for each player from play-by-play event data collected during matches.
- Europe > Spain > Galicia > Madrid (0.04)
- Europe > Netherlands (0.04)
- Europe > Italy > Lazio (0.04)
- (2 more...)
- Leisure & Entertainment > Sports > Soccer (1.00)
- Leisure & Entertainment > Sports > Football (1.00)
Distinguishing among the 50 shades of artificial intelligence
It has been my experience that whenever a new craze appears in the field of information technology (IT), the industry begins to have varying levels of flirtation with the concept, and each firm's executives try to best other firms by jumping on the bandwagon and then proceeding to make wide-ranging pronouncements about how they are using the new craze to transform their industry. Just a few years ago, the craze was to have a global delivery centre in India, either through an outsourcing relationship with an IT service provider or by tapping directly into the technology labour pool in India. This was starkly obvious to me when we hosted more than one Western client at an industry or service provider event--the first metrics they gauged each other by while sizing up where each stood on the totem pole were usually: "number of people in India" and "number of trips taken to India". Today, this chatter has moved on to topics around automation, artificial intelligence (AI), machine learning (ML), deep learning (DL) and blockchain. The irony is that many of these disciplines are actually quite old, but they have only now become fashionable catchphrases.
Evaluating Methods for Distinguishing Between Human-Readable Text and Garbled Text
Henderson, Jette L. (The University of Texas at Austin) | Frazee, Daniel J. (The University of Texas at Austin) | Siegel, Nick P. (The University of Texas at Austin) | Martin, Cheryl E. (The University of Texas at Austin) | Liu, Alexander Y. (The University of Texas at Austin)
In some cybersecurity applications, it is useful to differenti- ate between human-readable text and garbled text (e.g., en- coded or encrypted text). Automated methods are necessary for performing this task on large volumes of data. Which method is best is an open question that depends on the spe- cific problem context. In this paper, we explore this open question via empirical tests of many automated categoriza- tion methods for differentiating human-readable versus gar- bled text under a variety of conditions (e.g., different class priors, different problem contexts, concept drift, etc.). The results indicate that the best approaches tend to be either variants of naïve Bayes or classifiers that use low- dimensional, structural features. The results also indicate that concept drift is one of the most problematic issues when classifying garbled text.