We tackle the problem of novel class discovery and localization (NCDL) .

Implicit bias refers to automatic or spontaneous mental processes that shape perceptions, judgments, and behaviors based on social categories such as race, gender, or age [Greenwald and Lai, 2020, Payne and Gawronski, 2010]. Implicit biases often operate rapidly and with high efficiency, requiring minimal cognitive resources while influencing judgments through the automatic activation of stored information about social groups [Melnikoff and Bargh, 2018, Bargh and Williams, 2006, Fazio et al., 1986]. This efficiency in processing allows implicit biases to operate even under conditions of limited attention or cognitive load. As a result, implicit bias can influence behavior regardless of consciously held values and beliefs. Research demonstrates that implicit bias significantly relates to real-world outcomes, with researchers describing a potential role of implicit bias in domains such as employment [Agerström and Rooth, 2011], healthcare [FitzGerald and Hurst, 2017], and criminal justice [Spencer et al., 2016].

Growing evidence suggests that the brain uses an "attention schema" to monitor, predict, and help control attention. It has also been suggested that an attention schema improves social intelligence by allowing one person to better predict another. Given their potential advantages, attention schemas have been increasingly tested in machine learning. Here we test small deep learning networks to determine how the addition of an attention schema may affect performance on a range of tasks. First, we found that an agent with an attention schema is better at judging or categorizing the attention states of other agents. Second, we found that an agent with an attention schema develops a pattern of attention that is easier for other agents to judge and categorize. Third, we found that in a joint task where two agents paint a scene together and must predict each other's behavior for best performance, adding an attention schema improves that performance. Finally, we find that the performance improvements caused by an attention schema are not a non-specific result of an increase in network complexity. Not all performance, on all tasks, is improved. Instead, improvement is specific to "social" tasks involving judging, categorizing, or predicting the attention of other agents. These results suggest that an attention schema may be useful in machine learning for improving cooperativity and social behavior.

Last spring, a graduate student in social anthropology--let's call him Chris--sat down at his laptop and asked ChatGPT for help with a writing assignment. He pasted a few thousand words, a mix of rough summaries and jotted-down bullet points, into the text box that serves as ChatGPT's interface. "Here's my entire exam," he wrote. "Don't edit it, I will tell you what to do after you've read it." Chris was tackling a difficult paper about perspectivism, which is the anthropological principle that one's perspective inevitably shapes the observations one makes and the knowledge one acquires.

Is it possible to crowdsource categorization? Amongst the challenges: (a) each worker has only a partial view of the data, (b) different workers may have different clustering criteria and may produce different numbers of categories, (c) the underlying category structure may be hierarchical. We propose a Bayesian model of how workers may approach clustering and show how one may infer clusters / categories, as well as worker parameters, using this model. Our experiments, carried out on large collections of images, suggest that Bayesian crowdclustering works well and may be superior to single-expert annotations.

Social science NLP tasks, such as emotion or humor detection, are required to capture the semantics along with the implicit pragmatics from text, often with limited amounts of training data. Instruction tuning has been shown to improve the many capabilities of large language models (LLMs) such as commonsense reasoning, reading comprehension, and computer programming. However, little is known about the effectiveness of instruction tuning on the social domain where implicit pragmatic cues are often needed to be captured. We explore the use of instruction tuning for social science NLP tasks and introduce Socialite-Llama -- an open-source, instruction-tuned Llama. On a suite of 20 social science tasks, Socialite-Llama improves upon the performance of Llama as well as matches or improves upon the performance of a state-of-the-art, multi-task finetuned model on a majority of them. Further, Socialite-Llama also leads to improvement on 5 out of 6 related social tasks as compared to Llama, suggesting instruction tuning can lead to generalized social understanding. All resources including our code, model and dataset can be found through bit.ly/socialitellama.

Multi-modal Large Language Models (MLLMs) have shown remarkable capabilities in various multi-modal tasks. Nevertheless, their performance in fine-grained image understanding tasks is still limited. To address this issue, this paper proposes a new framework to enhance the fine-grained image understanding abilities of MLLMs. Specifically, we present a new method for constructing the instruction tuning dataset at a low cost by leveraging annotations in existing datasets. A self-consistent bootstrapping method is also introduced to extend existing dense object annotations into high-quality referring-expression-bounding-box pairs. These methods enable the generation of high-quality instruction data which includes a wide range of fundamental abilities essential for fine-grained image perception. Moreover, we argue that the visual encoder should be tuned during instruction tuning to mitigate the gap between full image perception and fine-grained image perception. Experimental results demonstrate the superior performance of our method. For instance, our model exhibits a 5.2% accuracy improvement over Qwen-VL on GQA and surpasses the accuracy of Kosmos-2 by 24.7% on RefCOCO_val. We also attain the top rank on the leaderboard of MMBench. This promising performance is achieved by training on only publicly available data, making it easily reproducible. The models, datasets, and codes are publicly available at https://github.com/SY-Xuan/Pink.

The invariance of an objects' identity as it transformed over time provides a powerful cue for perceptual learning. We present an un(cid:173) supervised learning procedure which maximizes the mutual infor(cid:173) mation between the representations adopted by a feed-forward net(cid:173) work at consecutive time steps. We demonstrate that the network can learn, entirely unsupervised, to classify an ensemble of several patterns by observing pattern trajectories, even though there are abrupt transitions from one object to another between trajecto(cid:173) ries. The same learning procedure should be widely applicable to a variety of perceptual learning tasks.

As we continue to accumulate digital photos on our devices, it can be challenging to keep them organized and easy to find. But artificial intelligence (AI) has made things easier by enabling a wide range of intelligent organization features. AI-powered photo organizers use machine learning algorithms to automatically tag, sort, and categorize photos based on their content, date, location, and other factors. These intelligent tools are becoming essential in the digital age, allowing us to quickly locate specific photos and share them with ease. One of the best AI-powered photo organizers on the market is PhotoPrism, an app that helps users manage and organize their digital photo collection more efficiently and effectively.

Non-rigid point set registration has been used in a wide range of computer vision applications such as human movement tracking, medical image analysis, three dimensional (3D) object reconstruction and is a very challenging task. It has two fundamental tasks. One is to find correspondences between two or more point sets and another is to transform a point set so that it aligns with other point sets. There has been significant progress in the past two decades in the non-rigid registration field but it still has major challenges and is an active research area in the computer vision and pattern recognition community. In this review, we present a survey of non-rigid point set registration.