weyl
A Short Note on Upper Bounds for Graph Neural Operator Convergence Rate
ABSTRACT Graphons, as limits of graph sequences, provide a framework for analyzing the asymptotic behavior of graph neural operators. Spectral convergence of sampled graphs to graphons yields operator-level convergence rates, enabling transferability analyses of GNNs. This note summarizes known bounds under no assumptions, global Lipschitz continuity, and piecewise-Lipschitz continuity, highlighting tradeoffs between assumptions and rates, and illustrating their empirical tightness on synthetic and real data. Index T erms-- graph neural operator, graphon, convergence rates, graph neural networks, transferability 1. INTRODUCTION Graph neural networks (GNNs) are widely used in drug discovery [1, 2], social networks [3, 4], recommendation systems [5], and NLP [6, 7, 8]. GNNs operate on graph-structured data via message passing and aggregation [9], but training on large graphs is computationally expensive.
Perturbation Analysis of Singular Values in Concatenated Matrices
Concatenating matrices is a common technique for uncovering shared structures in data through singular value decomposition (SVD) and low-rank approximations. The fundamental question arises: How does the singular value spectrum of the concatenated matrix relate to the spectra of its individual components? In the present work, we develop a perturbation technique that extends classical results such as Weyl's inequality to concatenated matrices. We setup analytical bounds that quantify stability of singular values under small perturbations in submatrices. The results demonstrate that if submatrices are close in a norm, dominant singular values of the concatenated matrix remain stable enabling controlled trade-offs between accuracy and compression. These provide a theoretical basis for improved matrix clustering and compression strategies with applications in the numerical linear algebra, signal processing, and data-driven modeling.
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks
Zhao, Jim, Singh, Sidak Pal, Lucchi, Aurelien
The Gauss-Newton (GN) matrix plays an important role in machine learning, most evident in its use as a preconditioning matrix for a wide family of popular adaptive methods to speed up optimization. Besides, it can also provide key insights into the optimization landscape of neural networks. In the context of deep neural networks, understanding the GN matrix involves studying the interaction between different weight matrices as well as the dependencies introduced by the data, thus rendering its analysis challenging. In this work, we take a first step towards theoretically characterizing the conditioning of the GN matrix in neural networks. We establish tight bounds on the condition number of the GN in deep linear networks of arbitrary depth and width, which we also extend to two-layer ReLU networks. We expand the analysis to further architectural components, such as residual connections and convolutional layers. Finally, we empirically validate the bounds and uncover valuable insights into the influence of the analyzed architectural components.
Taiwan's Digital Minister Has an Ambitious Plan to Align Tech With Democracy
Audrey Tang, Taiwan's 43-year-old minister of digital affairs, has a powerful effect on people. At a panel discussion at Northeastern University in Boston, 20-year-old student Diane Grant is visibly moved, describing Tang's talk as the best she's been to in her undergraduate career. Later that day, a German tourist recognizes Tang leaving the Boston Museum of Science and requests a photo, saying she's "starstruck." At the Massachusetts Institute of Technology, a trio of world-leading economists bashfully ask Tang to don a baseball cap emblazoned with the name of their research center and pose for a group photo. Political scientist and former gubernatorial candidate Danielle Allen, confesses to Tang that, although others often tell her that she is a source of inspiration to them, she rarely feels inspired by others.
Efficient Learning of Quantum States Prepared With Few Non-Clifford Gates II: Single-Copy Measurements
Grewal, Sabee, Iyer, Vishnu, Kretschmer, William, Liang, Daniel
Recent work has shown that $n$-qubit quantum states output by circuits with at most $t$ single-qubit non-Clifford gates can be learned to trace distance $\epsilon$ using $\mathsf{poly}(n,2^t,1/\epsilon)$ time and samples. All prior algorithms achieving this runtime use entangled measurements across two copies of the input state. In this work, we give a similarly efficient algorithm that learns the same class of states using only single-copy measurements.
We need to create guardrails for AI
Please use the sharing tools found via the share button at the top or side of articles. Subscribers may share up to 10 or 20 articles per month using the gift article service. More information can be found here. What if the only thing you could truly trust was something or someone close enough to physically touch? That may be the world into which AI is taking us. A group of Harvard academics and artificial intelligence experts has just launched a report aimed at putting ethical guardrails around the development of potentially dystopian technologies such Microsoft-backed OpenAI's seemingly sentient chatbot, which debuted in a new and "improved" (depending on your point of view) version, GPT-4, last week.
What if people were paid for their data?
Jennifer Lyn Morone, an American artist, thinks this is the state in which most people now live. To get free online services, she laments, they hand over intimate information to technology firms. "Personal data are much more valuable than you think," she says. To highlight this sorry state of affairs, Ms Morone has resorted to what she calls "extreme capitalism": she registered herself as a company in Delaware in an effort to exploit her personal data for financial gain. She created dossiers containing different subsets of data, which she displayed in a London gallery in 2016 and offered for sale, starting at £100 ($135). The entire collection, including her health data and social-security number, can be had for £7,000.
Can Big Tech Companies Find A Way To Reward Users For Their Data?
Personal data is the "new oil," in the words of The Economist. And the mining of online data is just the start. Increasingly, the offline world is also a data gold mine. As everything from cars to power plants are added to the internet of things, we're creating additional petabytes of data-rich resources. Companies can use this data to train algorithms that run new types of services, including traffic directions, automated transport networks, and factories that require only a few humans to operate.