Goto

Collaborating Authors

 Technology


k-Median Clustering via Metric Embedding: Towards Better Initialization with Differential Privacy

Neural Information Processing Systems

We propose a new initialization scheme for the k-median problem in the general metric space (e.g., discrete space induced by graphs), based on the construction of metric embedding tree structure of the data. We propose a novel and efficient search algorithm which finds initial centers that can be used subsequently for the local search algorithm. The so-called HST initialization method can produce initial centers achieving lower error than those from another popular method k-median++, also with higher efficiency when k is not too small. Our HST initialization are then extended to the setting of differential privacy (DP) to generate private initial centers. We show that the error of applying DP local search followed by our private HST initialization improves prior results on the approximation error, and approaches the lower bound within a small factor. Experiments demonstrate the effectiveness of our proposed methods.





I)j(II)deoi.52 tt

Neural Information Processing Systems

We present a new dataset condensation framework termed Squeeze (), Recover () and Relabel () (SRe2L) that decouples the bilevel optimization of model and architectures synthetic and data image during resolutions training, for to ef handle ficient dataset varying condensation.


Tailoring Self-Attention for Graph via Rooted Subtrees

Neural Information Processing Systems

Attention mechanisms have made significant strides in graph learning, yet they still exhibit notable limitations: local attention faces challenges in capturing long-range information due to the inherent problems of the message-passing scheme, while global attention cannot reflect the hierarchical neighborhood structure and fails to capture fine-grained local information. In this paper, we propose a novel multihop graph attention mechanism, named Subtree Attention (STA), to address the aforementioned issues. STA seamlessly bridges the fully-attentional structure and the rooted subtree, with theoretical proof that STA approximates the global attention under extreme settings.


ATechnical Lemmas

Neural Information Processing Systems

The proof is an induction on k. Consider the general case p2k+1. It is easy to see that g (x) = ex p2k(x) and g (x) = ex p2k 1(x). By the induction hypothesis, g 0 and therefore g is convex. Thus, the minimum of g is given by its stationary points. It is easy to observe that x = 0 is indeed a stationary point. Thus, minx R g(x) = g(0) = 0, which finishes the proof.



Alignment with human representations supports robust few-shot learning

Neural Information Processing Systems

Should we care whether AI systems have representations of the world that are similar to those of humans? We provide an information-theoretic analysis that suggests that there should be a U-shaped relationship between the degree of representational alignment with humans and performance on few-shot learning tasks. We confirm this prediction empirically, finding such a relationship in an analysis of the performance of 491 computer vision models. We also show that highly-aligned models are more robust to both natural adversarial attacks and domain shifts. Our results suggest that human alignment is often a sufficient, but not necessary, condition for models to make effective use of limited data, be robust, and generalize well.


e8da56eb93676e8f60ed2b696e44e7dc-Supplemental-Conference.pdf

Neural Information Processing Systems

The goal location is small region around (20,20). In each task, S0 was a set of arm con gurations establishing contact with the 539 end-effector, the 6-DoF change in stiffness, and 1-DoF gripper state. The fraction of start states in S0 that lead to success 557 IVF, classi er). The result of that execution is recorded as 552 Algorithm 1 is the pseudocode used for the experiments described in Section 4.1. Episodes last a maximum of 1000 steps.