unmasking
Unmasking the Canvas: A Dynamic Benchmark for Image Generation Jailbreaking and LLM Content Safety
Nair, Variath Madhupal Gautham, Dantuluri, Vishal Varma
Existing large language models (LLMs) are advancing rapidly and produce outstanding results in image generation tasks, yet their content safety checks remain vulnerable to prompt-based jailbreaks. Through preliminary testing on platforms such as ChatGPT, MetaAI, and Grok, we observed that even short, natural prompts could lead to the generation of compromising images ranging from realistic depictions of forged documents to manipulated images of public figures. We introduce Unmasking the Canvas (UTC Benchmark; UTCB), a dynamic and scalable benchmark dataset to evaluate LLM vulnerability in image generation. Our methodology combines structured prompt engineering, multilingual obfuscation (e.g., Zulu, Gaelic, Base64), and evaluation using Groq-hosted LLaMA-3. The pipeline supports both zero-shot and fallback prompting strategies, risk scoring, and automated tagging. All generations are stored with rich metadata and curated into Bronze (non-verified), Silver (LLM-aided verification), and Gold (manually verified) tiers. UTCB is designed to evolve over time with new data sources, prompt templates, and model behaviors. Warning: This paper includes visual examples of adversarial inputs designed to test model safety. All outputs have been redacted to ensure responsible disclosure.
Unmasking the Uniqueness: A Glimpse into Age-Invariant Face Recognition of Indigenous African Faces
Ajewole, Fakunle, Akinyemi, Joseph Damilola, Ladoja, Khadijat Tope, Onifade, Olufade Falade Williams
The task of recognizing the age-separated faces of an individual, Age-Invariant Face Recognition (AIFR), has received considerable research efforts in Europe, America, and Asia, compared to Africa. Thus, AIFR research efforts have often under-represented/misrepresented the African ethnicity with non-indigenous Africans. This work developed an AIFR system for indigenous African faces to reduce the misrepresentation of African ethnicity in facial image analysis research. We adopted a pre-trained deep learning model (VGGFace) for AIFR on a dataset of 5,000 indigenous African faces (FAGE\_v2) collected for this study. FAGE\_v2 was curated via Internet image searches of 500 individuals evenly distributed across 10 African countries. VGGFace was trained on FAGE\_v2 to obtain the best accuracy of 81.80\%. We also performed experiments on an African-American subset of the CACD dataset and obtained the best accuracy of 91.5\%. The results show a significant difference in the recognition accuracies of indigenous versus non-indigenous Africans.
DEMASQ: Unmasking the ChatGPT Wordsmith
Kumari, Kavita, Pegoraro, Alessandro, Fereidooni, Hossein, Sadeghi, Ahmad-Reza
The potential misuse of ChatGPT and other Large Language Models (LLMs) has raised concerns regarding the dissemination of false information, plagiarism, academic dishonesty, and fraudulent activities. Consequently, distinguishing between AI-generated and human-generated content has emerged as an intriguing research topic. However, current text detection methods lack precision and are often restricted to specific tasks or domains, making them inadequate for identifying content generated by ChatGPT. In this paper, we propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. Our method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machine-generated content and (ii) the alterations made by humans to evade previous detection methods. DEMASQ is an energy-based detection model that incorporates novel aspects, such as (i) optimization inspired by the Doppler effect to capture the interdependence between input text embeddings and output labels, and (ii) the use of explainable AI techniques to generate diverse perturbations. To evaluate our detector, we create a benchmark dataset comprising a mixture of prompts from both ChatGPT and humans, encompassing domains such as medical, open Q&A, finance, wiki, and Reddit. Our evaluation demonstrates that DEMASQ achieves high accuracy in identifying content generated by ChatGPT.
Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Data
Azizmalayeri, Mohammad, Abu-Hanna, Ameen, Ciná, Giovanni
Despite their success, Machine Learning (ML) models do not generalize effectively to data not originating from the training distribution. To reliably employ ML models in real-world healthcare systems and avoid inaccurate predictions on out-of-distribution (OOD) data, it is crucial to detect OOD samples. Numerous OOD detection approaches have been suggested in other fields - especially in computer vision - but it remains unclear whether the challenge is resolved when dealing with medical tabular data. To answer this pressing need, we propose an extensive reproducible benchmark to compare different methods across a suite of tests including both near and far OODs. Our benchmark leverages the latest versions of eICU and MIMIC-IV, two public datasets encompassing tens of thousands of ICU patients in several hospitals. We consider a wide array of density-based methods and SOTA post-hoc detectors across diverse predictive architectures, including MLP, ResNet, and Transformer. Our findings show that i) the problem appears to be solved for far-OODs, but remains open for near-OODs; ii) post-hoc methods alone perform poorly, but improve substantially when coupled with distance-based mechanisms; iii) the transformer architecture is far less overconfident compared to MLP and ResNet.
Unmasking the giant: A comprehensive evaluation of ChatGPT's proficiency in coding algorithms and data structures
Arefin, Sayed Erfan, Heya, Tasnia Ashrafi, Al-Qudah, Hasan, Ineza, Ynes, Serwadda, Abdul
The transformative influence of Large Language Models (LLMs) is profoundly reshaping the Artificial Intelligence (AI) technology domain. Notably, ChatGPT distinguishes itself within these models, demonstrating remarkable performance in multi-turn conversations and exhibiting code proficiency across an array of languages. In this paper, we carry out a comprehensive evaluation of ChatGPT's coding capabilities based on what is to date the largest catalog of coding challenges. Our focus is on the python programming language and problems centered on data structures and algorithms, two topics at the very foundations of Computer Science. We evaluate ChatGPT for its ability to generate correct solutions to the problems fed to it, its code quality, and nature of run-time errors thrown by its code. Where ChatGPT code successfully executes, but fails to solve the problem at hand, we look into patterns in the test cases passed in order to gain some insights into how wrong ChatGPT code is in these kinds of situations. To infer whether ChatGPT might have directly memorized some of the data that was used to train it, we methodically design an experiment to investigate this phenomena. Making comparisons with human performance whenever feasible, we investigate all the above questions from the context of both its underlying learning models (GPT-3.5 and GPT-4), on a vast array sub-topics within the main topics, and on problems having varying degrees of difficulty.
How Deepfake Works: Unmasking the Technology with Applications and 7 Ways to Detect
These days we can see tons of images and videos of celebrities, such as A video in which Tom Cruise is talking about Politics and Trump is talking about Hollywood movies. Before diving into how this is possible, let's talk about What is Deepfake and How do deepfakes work? Deepfake is a portmanteau of "deep learning" and "fake". It means using artificial intelligence (AI) techniques to make fake audio, video, or image content that looks and sounds real. Machine learning algorithms are used to make these very realistic fakes.
Unmasking the Black Box Problem of Machine Learning - InformationWeek
Financial and banking services company Standard Chartered turned to a model intelligence platform to get a clearer picture of how its algorithms make decisions on customer data. How machine learning comes to conclusions and produces results can be a bit mysterious, even to the teams that develop the algorithms that drive them -- the so-called black box problem. Standard Chartered chose Truera to help it lift away some of the obscurity and potential biases that might affect results from its ML models. "Data scientists don't directly build the models," says Will Uppington, CEO and co-founder of Truera. "The machine learning algorithm is the direct builder of the model."
Unmasking the Black Box Problem of Machine Learning - InformationWeek
Financial and banking services company Standard Chartered turned to a model intelligence platform to get a clearer picture of how its algorithms make decisions on customer data. How machine learning comes to conclusions and produces results can be a bit mysterious, even to the teams that develop the algorithms that drive them -- the so-called black box problem. Standard Chartered chose Truera to help it lift away some of the obscurity and potential biases that might affect results from its ML models. "Data scientists don't directly build the models," says Will Uppington, CEO and co-founder of Truera. "The machine learning algorithm is the direct builder of the model."
Blockchain Technology & Artificial Intelligence -- Unmasking the Mystery at the Heart of AI
Peanut butter and chocolate, Mick and Keith, Batman and Robin -- great partnerships -- successful marriages so to speak. Something magical happened when their paths merged, forging culinary, music and comic book history. When two technologies collide, the result is groundbreaking innovation -- healthcare and robotics, supply chain and distributed ledger technology, digital technology, and photography, 3D printing & healthcare. In business for an innovative idea to be implemented on a large scale, it has to solve a specific need and it needs to be able to be replicated at a reasonable cost. Though the field of artificial intelligence was born in the 1950s, it didn't really find mainstream popularity until the 1990s and early 2000s.