Large Language Model
Zero-shot recognition with unreliable attributes
Dinesh Jayaraman, Kristen Grauman
In principle, zero-shot learning makes it possible to train a recognition model simply by specifying the category's attributes. For example, with classifiers for generic attributes like striped and four-legged, one can construct a classifier for the zebra category by enumerating which properties it possesses--even without providing zebra training images. In practice, however, the standard zero-shot paradigm suffers because attribute predictions in novel images are hard to get right. We propose a novel random forest approach to train zero-shot models that explicitly accounts for the unreliability of attribute predictions. By leveraging statistics about each attribute's error tendencies, our method obtains more robust discriminative models for the unseen classes. We further devise extensions to handle the few-shot scenario and unreliable attribute descriptions. On three datasets, we demonstrate the benefit for visual category learning with zero or few training examples, a critical domain for rare categories or categories defined on the fly.
Big Tech whistleblower's parents sue, sounding alarm over son's unexpected death
If you or someone you know is having thoughts of suicide, please contact the Suicide & Crisis Lifeline at 988 or 1-800-273-TALK (8255). The parents of a young California tech whistleblower whose 2024 death was ruled a suicide are now suing the City and County of San Francisco, alleging they violated public records laws by refusing to fulfill their requests for information about their son's death. Suchir Balaji, 26, was an employee at OpenAI, the artificial intelligence company behind ChatGPT, at the time of his Nov. 26, 2024, death. A San Francisco County medical examiner concluded the next day he died from a self-inflicted gunshot wound inside his apartment. "In the two-plus months since their son's passing, Petitioners and their counsel have been stymied at every turn as they have sought more information about the cause of and circumstances surrounding Suchir's tragic death. This petition, they hope, is the beginning of the end of that obstruction," the lawsuit states.
u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality
While audio-visual speech models can yield superior performance and robustness compared to audio-only models, their development and adoption are hindered by the lack of labeled and unlabeled audio-visual data and the cost to deploy one model per modality. In this paper, we present u-HuBERT, a self-supervised pre-training framework that can leverage both multimodal and unimodal speech with a unified masked cluster prediction objective. By utilizing modality dropout during pre-training, we demonstrate that a single fine-tuned model can achieve performance on par or better than the state-of-the-art modality-specific models. Moreover, our model fine-tuned only on audio can perform well with audio-visual and visual speech input, achieving zero-shot modality generalization for multiple speech processing tasks.
Fox News AI Newsletter: Bill would ban DeepSeek
'The Big Weekend Show' co-hosts discuss the impact of new artificial intelligence apps on national security and jobs. 'DEEPLY DISTURBING EVIDENCE': House lawmakers are introducing legislation Thursday to ban the Chinese AI DeepSeek from U.S. government devices, arguing that the software is "directly linked to the Chinese Communist Party" and poses a "five-alarm national security fire." AN AMERICAN TIKTOK: The CEO of AI startup Perplexity, Aravind Srinivas, confirmed his company's bid for TikTok U.S. and said the deal checks all the boxes for investors and President Donald Trump, including an ownership stake for the U.S. SEEING DOUBLE: Donny Osmond is bringing on a new co-star for his Las Vegas residency: himself. President Donald Trump speaks to reporters after signing a series of executive orders in the Oval Office of the White House on January 23, 2025, in Washington, D.C. 'AI POWERHOUSE': The White House is opening its artificial intelligence plan up to Americans to contribute policy ideas to the Trump administration to ensure that the United States is "the undeniable leader" in AI technology. PEDAL TO THE METAL: For a commander on the battlefield, a split second of decision advantage can determine the difference between victory and defeat.
Review for NeurIPS paper: Attribute Prototype Network for Zero-Shot Learning
Weaknesses: Novelty 1- The proposed model is mainly building on previous ideas: [8] for learning prototypes, [15] for decorrelation and sharing, [52] for localization compactness, and [7] for score calibration. This renders the technical novelty to be somewhat limited. Nonetheless, I find the employment of these ideas together for attribute localization and ZSL is quite interesting and seems to lead to consistent good performance. Model: 2- It seems that the model uses continuous attributes. This type of attributes is usually obtained averaging the image-level binary attributes for each class which is expensive to obtain.
Review for NeurIPS paper: Attribute Prototype Network for Zero-Shot Learning
The authors satisfactorily addressed the concerns of the reviewers regarding novelty, comparisons to related work, and visualizations of mixed-attribute localization. As such, it is felt that this paper would be a good contribution to the ZSL field. Please incorporate the exposition, clarifications, and additional experiments from the rebuttal into the main text.
20 million OpenAI users hacked? Here's how to stay safe, just in case
Have you ever tried ChatGPT? You may want to take a quick moment to freshen up your account's security. A Russian hacker is claiming to have login data for over 20 million OpenAI users--and the information includes email addresses and passwords. On Friday, samples of OpenAI logins emerged on the dark web, along with an offer to sell the full trove of data. Currently, OpenAI says it has not yet found evidence of compromised systems (as per The Independent).
Review for NeurIPS paper: Uncertainty-Aware Learning for Zero-Shot Semantic Segmentation
This paper presents an interesting two-branch framework to address the zero-shot semantic segmentation problem. The approach is one of the first to utilize uncertainty modeling, both at the pixel and image level to model label/observation noise, in the zero-shot setting. The reviewers appreciated the approach and the results, but expressed some concerns about clarity of the method (esp. The rebuttal addressed some of these concerns, and the clarifications should be added to the camera-ready version. Overall, this paper has a nice contribution to the sub-field that would be of interest to the community.
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
CLIP yielded impressive results on zero-shot transfer learning tasks and is considered as a foundation model like BERT or GPT3. CLIP vision models that have a rich representation are pre-trained using the InfoNCE objective and natural language supervision before they are fine-tuned on particular tasks. Though CLIP excels at zero-shot transfer learning, it suffers from an explaining away problem, that is, it focuses on one or few features, while neglecting other relevant features. This problem is caused by insufficiently extracting the covariance structure in the original multi-modal data. We suggest to use modern Hopfield networks to tackle the problem of explaining away. Their retrieved embeddings have an enriched covariance structure derived from co-occurrences of features in the stored embeddings.