Goto

Collaborating Authors

 Technology


Differentiable Unsupervised Feature Selection based on a Gated Laplacian - Supplementary Materials

Neural Information Processing Systems

It is important to properly tune the kernel scale/bandwidth σb, which determines its scale of connectivity. Several studies have proposed schemes for tuning σb, see for example [10, 3, 12, 5]. Here, we focus on two schemes, a global bandwidth and a local bandwidth. The local bandwidth proposed in [12], involves setting a local-scale σi for each data point xi,i= 1,...,n. The scale is chosen using the L1 distance from the k-th nearest neighbor of the point xi.


Differentiable Unsupervised Feature Selection based on a Gated Laplacian

Neural Information Processing Systems

Scientific observations may consist of a large number of variables (features). Selecting a subset of meaningful features is often crucial for identifying patterns hidden in the ambient space. In this paper, we present a method for unsupervised feature selection, and we demonstrate its advantage in clustering, a common unsupervised task. We propose a differentiable loss that combines a graph Laplacian-based score that favors low-frequency features with a gating mechanism for removing nuisance features. Our method improves upon the naive graph Laplacian score by replacing it with a gated variant computed on a subset of low-frequency features. We identify this subset by learning the parameters of continuously relaxed Bernoulli variables, which gate the entire feature space. We mathematically motivate the proposed approach and demonstrate that it is crucial to compute the graph Laplacian on the gated inputs rather than on the full feature space in the high noise regime. Using several real-world examples, we demonstrate the efficacy and advantage of the proposed approach over leading baselines.


Chornobyl at 40: Settlers and horses survive Russian drones, contamination

Al Jazeera

What are Russia's gains from the Iran war? 'We are not losers; we are winners' But the calm is deceptive. Two soldiers scour the skies, hands firmly gripping anti-aircraft guns mounted on pick-up trucks parked on a small, dilapidated bridge on a tributary of the Pripyat River. Danger is all around, both in the surrounding land, which still carries the legacy of the 1986 Chornobyl nuclear disaster, with pockets of intense radioactive contamination, and above, where Russian drones and missiles launched from just across the border in Belarus, a short distance to the north, regularly pass overhead. The area is known as the Chornobyl Exclusion Zone (CEZ), a restricted area of approximately 30km (19 miles) in diameter, comparable in size to Luxembourg, established to contain the spread of contamination. Since Russia launched its full-scale invasion of Ukraine on February 24, 2022, briefly occupying the CEZ and the surrounding area, large swaths of it have become militarised, adding another layer of restriction to an already tightly controlled and hazardous environment. Yet despite the CEZ's many dangers, four decades on from the Chornobyl disaster, small communities of scientists, elderly returnees and soldiers have carved out lives among its abandoned buildings, while wildlife thrives in the surrounding forests.


0b9e57c46de934cee33b0e8d1839bfc2-Supplemental.pdf

Neural Information Processing Systems

We use Law(X) to denote the distribution of random variable X. When ν is a probability distribution for over set Ω and Ais a subset of Ω, we use ν(A) to denote the probability that the random variable X belongs to A, when X is sampled from distribution ν. Similarly, the marginal distribution on the next N random variables for fγ,r#ν is fγ,r#ν2. We thus proved equation (22) and Lemma 3 is proved. Lemma 4 Suppose Z1( |A) and Z2( |A) are two conditional distribution with range RN, and for all values of a Ω, Wp(Law(Z1(a)),Law(Z2(a))) c (23) where Z1(a) Z1( |A= a), and Z2(a) Z2( |A= a).



Few-Shot Audio-Visual Learning of Environment Acoustics Supplementary Material

Neural Information Processing Systems

In this supplementary material we provide additional details about: Video (with audio) for qualitative illustration of our task and qualitative evaluation of our model predictions (Sec. Evaluation of the impact of the query source location on our model's prediction quality for a fixed receiver (Sec. Moreover, we qualitatively demonstrate our model's prediction quality by comparing the predictions with the ground truths, both at the RIR level and in terms of perceptual similarity when the RIRs are convolved with real-world monaural sounds, like speech and music. We also analyze common failure cases for our model (Sec. Please use headphones to hear the spatial audio correctly.



Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Neural Information Processing Systems

When answering a question, humans utilize the information available across different modalities to synthesize a consistent and complete chain of thought (CoT). This process is normally a black box in the case of deep learning models like large-scale language models. Recently, science question benchmarks have been used to diagnose the multi-hop reasoning ability and interpretability of an AI system. However, existing datasets fail to provide annotations for the answers, or are restricted to the textual-only modality, small scales, and limited domain diversity. To this end, we present Science Question Answering (SCIENCEQA), a new benchmark that consists of 21k multimodal multiple choice questions with diverse science topics and annotations of their answers with corresponding lectures and explanations. We further design language models to learn to generate lectures and explanations as the chain of thought (CoT) to mimic the multi-hop reasoning process when answering SCIENCEQA questions. SCIENCEQA demonstrates the utility of CoT in language models, as CoT improves the question answering performance by 1.20% in fewshot GPT-3 and 3.99% in fine-tuned UnifiedQA. We also explore the upper bound for models to leverage explanations by feeding those in the input; we observe that it improves the few-shot performance of GPT-3 by 18.96%. Our analysis further shows that language models, similar to humans, benefit from explanations to learn from fewer data and achieve the same performance with just 40% of the data.1


Play to Grade: Testing Coding Games as Classifying Markov Decision Process

Neural Information Processing Systems

Contemporary coding education often presents students with the task of developing programs that have user interaction and complex dynamic systems, such as mouse based games. While pedagogically compelling, there are no contemporary autonomous methods for providing feedback. Notably, interactive programs are impossible to grade by traditional unit tests.


Learning Rule-Induced Subgraph Representations for Inductive Relation Prediction

Neural Information Processing Systems

Inductive relation prediction (IRP)--where entities can be different during training and inference--has shown great power for completing evolving knowledge graphs. Existing works mainly focus on using graph neural networks (GNNs) to learn the representation of the subgraph induced from the target link, which can be seen as an implicit rule-mining process to measure the plausibility of the target link. However, these methods cannot differentiate the target link and other links during message passing, hence the final subgraph representation will contain irrelevant rule information to the target link, which reduces the reasoning performance and severely hinders the applications for real-world scenarios. To tackle this problem, we propose a novel single-source edge-wise GNN model to learn the Rule-inducEd Subgraph represenTations (REST), which encodes relevant rules and eliminates irrelevant rules within the subgraph. Specifically, we propose a single-source initialization approach to initialize edge features only for the target link, which guarantees the relevance of mined rules and target link. Then we propose several RNN-based functions for edge-wise message passing to model the sequential property of mined rules. REST is a simple and effective approach with theoretical support to learn the rule-induced subgraph representation. Moreover, REST does not need node labeling, which significantly accelerates the subgraph preprocessing time by up to 11.66 . Experiments on inductive relation prediction benchmarks demonstrate the effectiveness of our REST2.