Technology
DISCS: ABenchmark for Discrete Sampling
Sampling in discrete spaces, with critical applications in simulation and opti-1 mization, has recently been boosted by significant advances in gradient-based2 approaches that exploit modern accelerators like GPUs. However, two key chal-3 lenges hinder the further research progress in discrete sampling. First, since there4 is no consensus on experimental settings, the empirical results in different research5 papers are often not comparable. Secondly, implementing samplers and target6 distributions often requires a nontrivial amount of effort in terms of calibration,7 parallelism, and evaluation. To tackle these challenges, we propose DISCS (DIS-8 Crete Sampling), a tailored package and benchmark that supports unified and9 efficient implementation and evaluations for discrete sampling in three types of10 tasks: sampling for classical graphical models, combinatorial optimization, and11 energy based generative models. Throughout the comprehensive evaluations in12 DISCS, we acquired new insights into scalability, design principles for proposal13 distributions, and lessons for adaptive sampling design.
Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation
Many fine-grained classification tasks, like rare animal identification, have limited training data and consequently classifiers trained on these datasets often fail to generalize to variations in the domain like changes in weather or location. As such, we explore how natural language descriptions of the domains seen in training data can be used with large vision models trained on diverse pretraining datasets to generate useful variations of the training data. We introduce ALIA (Automated Language-guided Image Augmentation), a method which utilizes large vision and language models to automatically generate natural language descriptions of a dataset's domains and augment the training data via language-guided image editing. To maintain data integrity, a model trained on the original dataset filters out minimal image edits and those which corrupt class-relevant information. The resulting dataset is visually consistent with the original training data and offers significantly enhanced diversity. We show that ALIA is able to surpasses traditional data augmentation and text-to-image generated data on fine-grained classification tasks, including cases of domain generalization and contextual bias. Code is available at https://github.com/lisadunlap/ALIA.
Reid Hoffman Thinks Doctors Should Ask AI for a Second Opinion
The LinkedIn cofounder now has an AI drug discovery startup--and thinks not asking chatbots for medical advice is "bordering on committing malpractice." Following a three-decade career at the helm of some of Silicon Valley's most powerful companies--cofounding LinkedIn and sitting on the boards of PayPal and OpenAI-- Reid Hoffman recently turned his attention to health care. Hoffman's startup, Manas AI, is building an AI engine that aims to fast-track the traditionally slow process of drug discovery for various cancers. Inspired by a dinner with renowned cancer physician Siddhartha Mukherjee, the company's cofounder and CEO, its mission statement is to "shift drug discovery from a decade-long process to one that takes a few years." But Hoffman's enthusiasm for generative AI, in particular, stretches far beyond novel drug targets and small molecules.
Meta in row after sacking workers who say they saw smart glasses users having sex
Meta is under pressure to explain why it cancelled a major contract with a company it was using to train AI, shortly after some of its Kenya-based workers alleged they had to view graphic content captured by Meta smart glasses. In February, workers at the company, Sama, told two Swedish newspapers they had witnessed glasses users going to the toilet and having sex . Less than two months later, Meta ended its contract with Sama, which Sama said would result in 1,108 workers being made redundant. Meta says it's because Sama did not meet its standards, a criticism Sama rejects. A Kenyan workers' organisation alleges Meta's decision was caused by the staff speaking out.
In Distribution via Discrete Diffusion
The widespread deployment of Graph Neural Networks (GNNs) sparks significant interest in their explainability, which plays a vital role in model auditing and ensuring trustworthy graph learning. The objective of GNN explainability is to discern the underlying graph structures that have the most significant impact on model predictions. Ensuring that explanations generated are reliable necessitates consideration of the in-distribution property, particularly due to the vulnerability of GNNs to out-of-distribution data. Unfortunately, prevailing explainability methods tend to constrain the generated explanations to the structure of the original graph, thereby downplaying the significance of the in-distribution property and resulting in explanations that lack reliability. To address these challenges, we propose D4Explainer, a novel approach that provides in-distribution GNN explanations for both counterfactual and model-level explanation scenarios. The proposed D4Explainer incorporates generative graph distribution learning into the optimization objective, which accomplishes two goals: 1) generate a collection of diverse counterfactual graphs that conform to the in-distribution property for a given instance, and 2) identify the most discriminative graph patterns that contribute to a specific class prediction, thus serving as model-level explanations. It is worth mentioning that D4Explainer is the first unified framework that combines both counterfactual and model-level explanations. Empirical evaluations conducted on synthetic and real-world datasets provide compelling evidence of the state-ofthe-art performance achieved by D4Explainer in terms of explanation accuracy, faithfulness, diversity, and robustness. 1
CoPriv: Network/Protocol Co-Optimization for Communication-Efficient Private Inference
Deep neural network (DNN) inference based on secure 2-party computation (2PC) can offer cryptographically-secure privacy protection but suffers from orders of magnitude latency overhead due to enormous communication. Previous works heavily rely on a proxy metric of ReLU counts to approximate the communication overhead and focus on reducing the ReLUs to improve the communication efficiency. However, we observe these works achieve limited communication reduction for state-of-the-art (SOTA) 2PC protocols due to the ignorance of other linear and non-linear operations, which now contribute to the majority of communication.