tnc
Causal Representation Learning for Generalisable Recommendation
Felekis, Yorgos, O'Riordan, Michael, Corcoll, Oriol, Gilligan-Lee, Ciarán M.
Predictive models trained on observational data often fail to generalise to the distributions they encounter when deployed, especially when the training data is a product of the system being optimised. Recommender systems are a canonical example: they are trained on interaction logs confounded by the deployed policy, past user behaviour, and platform filtering. As a result, the training distribution differs substantially from the candidate distribution scored at serving time, a gap that makes offline metrics unreliable predictors of online performance. We address the distribution shift problem with a method motivated by causal representation learning (CRL). We propose an information-theoretic disentanglement criterion and prove that its optimum depends only on the causal components of the input. We then derive a tractable variational lower bound that makes the criterion optimisable from finite observational data alone. The scope of our method is narrower than that of much of the CRL literature, in that we target better generalisation under distribution shift, not full identification of all latent causal factors. This narrower target is what makes the method practical, requiring only the existing confounded logs, applying to any standard supervised model, and adding no inference-time cost. Our headline evaluation is an A/B test with millions of users on Spotify, applied to a production ranker for personalised playlist generation. A capacity-matched CRL variant performed on par offline but delivered substantial online gains in listener engagement. Complementary evidence on the public KuaiRand recommendation dataset and a synthetic benchmark with known causal structure shows the same pattern: offline parity with baseline, gains under distribution shift. Across all three settings, adding our causal disentanglement objective yields meaningfully better out-of-distribution generalisation.
TDFormer: A Top-Down Attention-Controlled Spiking Transformer
Zhu, Zizheng, Yu, Yingchao, Zheng, Zeqi, Yu, Zhaofei, Jin, Yaochu
Traditional spiking neural networks (SNNs) can be viewed as a combination of multiple subnetworks with each running for one time step, where the parameters are shared, and the membrane potential serves as the only information link between them. However, the implicit nature of the membrane potential limits its ability to effectively represent temporal information. As a result, each time step cannot fully leverage information from previous time steps, seriously limiting the model's performance. Inspired by the top-down mechanism in the brain, we introduce TDFormer, a novel model with a top-down feedback structure that functions hierarchically and leverages high-order representations from earlier time steps to modulate the processing of low-order information at later stages. The feedback structure plays a role from two perspectives: 1) During forward propagation, our model increases the mutual information across time steps, indicating that richer temporal information is being transmitted and integrated in different time steps. 2) During backward propagation, we theoretically prove that the feedback structure alleviates the problem of vanishing gradients along the time dimension. We find that these mechanisms together significantly and consistently improve the model performance on multiple datasets. In particular, our model achieves state-of-the-art performance on ImageNet with an accuracy of 86.83%.
MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularization
Duan, Yue, Zhao, Zhen, Qi, Lei, Wang, Lei, Zhou, Luping, Shi, Yinghuan, Gao, Yang
The core issue in semi-supervised learning (SSL) lies in how to effectively leverage unlabeled data, whereas most existing methods tend to put a great emphasis on the utilization of high-confidence samples yet seldom fully explore the usage of low-confidence samples. In this paper, we aim to utilize low-confidence samples in a novel way with our proposed mutex-based consistency regularization, namely MutexMatch. Specifically, the high-confidence samples are required to exactly predict "what it is" by conventional True-Positive Classifier, while the low-confidence samples are employed to achieve a simpler goal -- to predict with ease "what it is not" by True-Negative Classifier. In this sense, we not only mitigate the pseudo-labeling errors but also make full use of the low-confidence unlabeled data by consistency of dissimilarity degree. MutexMatch achieves superior performance on multiple benchmark datasets, i.e., CIFAR-10, CIFAR-100, SVHN, STL-10, mini-ImageNet and Tiny-ImageNet. More importantly, our method further shows superiority when the amount of labeled data is scarce, e.g., 92.23% accuracy with only 20 labeled data on CIFAR-10. Our code and model weights have been released at https://github.com/NJUyued/MutexMatch4SSL.
Sustaining a national wonder with AI
The Mojave Desert in the southwestern United States is a vast landscape combining mountains and dried lake beds, forests, and wildflower fields. It has a rich cultural history, with 12,000-year-old archaeological sites, and it harbors protected species such golden eagles and desert tortoises, as well as its famous Joshua trees, some of which are 900-years old and 30-feet tall. But according to Lukas Agnew, senior consultant at Capgemini's insights and data practice, this unique environment is under threat. "Humans are increasingly using the desert for recreation, such as off-road dirt biking," he says. "While one bike might not cause a problem, over time, real damage is done."
Faster Rates of Differentially Private Stochastic Convex Optimization
In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) and provide excess population risks for some special classes of functions that are faster than the previous results of general convex and strongly convex functions. In the first part of the paper, we study the case where the population risk function satisfies the Tysbakov Noise Condition (TNC) with some parameter $\theta>1$. Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of $\tilde{O}((\frac{1}{\sqrt{n}}+\frac{\sqrt{d\log \frac{1}{\delta}}}{n\epsilon})^\frac{\theta}{\theta-1})$ for $(\epsilon, \delta)$-DP when $\theta\geq 2$, here $n$ is the sample size and $d$ is the dimension of the space. Then we address the inefficiency issue, improve the upper bounds by $\text{Poly}(\log n)$ factors and extend to the case where $\theta\geq \bar{\theta}>1$ for some known $\bar{\theta}$. Next we show that the excess population risk of population functions satisfying TNC with parameter $\theta>1$ is always lower bounded by $\Omega((\frac{d}{n\epsilon})^\frac{\theta}{\theta-1}) $ and $\Omega((\frac{\sqrt{d\log \frac{1}{\delta}}}{n\epsilon})^\frac{\theta}{\theta-1})$ for $\epsilon$-DP and $(\epsilon, \delta)$-DP, respectively. In the second part, we focus on a special case where the population risk function is strongly convex. Unlike the previous studies, here we assume the loss function is {\em non-negative} and {\em the optimal value of population risk is sufficiently small}. With these additional assumptions, we propose a new method whose output could achieve an upper bound of $O(\frac{d\log\frac{1}{\delta}}{n^2\epsilon^2}+\frac{1}{n^{\tau}})$ for any $\tau\geq 1$ in $(\epsilon,\delta)$-DP model if the sample size $n$ is sufficiently large.
K-Prototype Segmentation Analysis on Large-scale Ridesourcing Trip Data
Soria, J, Chen, Y, Stathopoulos, A
Shared mobility-on-demand services are expanding rapidly in cities around the world. As a prominent example, app-based ridesourcing is becoming an integral part of many urban transportation ecosystems. Despite the centrality, limited public availability of detailed temporal and spatial data on ridesourcing trips has limited research on how new services interact with traditional mobility options and how they impact travel in cities. Improving data-sharing agreements are opening unprecedented opportunities for research in this area. This study examines emerging patterns of mobility using recently released City of Chicago public ridesourcing data. The detailed spatio-temporal ridesourcing data are matched with weather, transit, and taxi data to gain a deeper understanding of ridesourcings role in Chicagos mobility system. The goal is to investigate the systematic variations in patronage of ride-hailing. K-prototypes is utilized to detect user segments owing to its ability to accept mixed variable data types. An extension of the K-means algorithm, its output is a classification of the data into several clusters called prototypes. Six ridesourcing prototypes are identified and discussed based on significant differences in relation to adverse weather conditions, competition with alternative modes, location and timing of use, and tendency for ridesplitting. The paper discusses implications of the identified clusters related to affordability, equity and competition with transit.
Quantum Compressed Sensing with Unsupervised Tensor Network Machine Learning
Ran, Shi-Ju, Sun, Zheng-Zhi, Fei, Shao-Ming, Su, Gang, Lewenstein, Maciej
We propose tensor-network compressed sensing (TNCS) for compressing and communicating classical information via the quantum states trained by the unsupervised tensor network (TN) machine learning. The main task of TNCS is to reconstruct as accurately as possible the full classical information from a generative TN state, by knowing as small part of the classical information as possible. In the applications to the datasets of hand-written digits and fashion images, we train the generative TN (matrix product state) by the training set, and show that the images in the testing set can be reconstructed from a small number of pixels. Related issues including the applications of TNCS to quantum encrypted communication are discussed.
The latest weapon in the fight against illegal fishing? Artificial intelligence
Facial recognition software is most commonly known as a tool to help police identify a suspected criminal by using machine learning algorithms to analyze his or her face against a database of thousands or millions of other faces. The larger the database, with a greater variety of facial features, the smarter and more successful the software becomes – effectively learning from its mistakes to improve its accuracy. Now, this type of artificial intelligence is starting to be used in fighting a specific but pervasive type of crime – illegal fishing. Rather than picking out faces, the software tracks the movement of fishing boats to root out illegal behavior. And soon, using a twist on facial recognition, it may be able to recognize when a boat's haul includes endangered and protected fish.
The latest weapon in the fight against illegal fishing? Artificial intelligence
Facial recognition software is most commonly known as a tool to help police identify a suspected criminal by using machine learning algorithms to analyze his or her face against a database of thousands or millions of other faces. The larger the database, with a greater variety of facial features, the smarter and more successful the software becomes – effectively learning from its mistakes to improve its accuracy. Now, this type of artificial intelligence is starting to be used in fighting a specific but pervasive type of crime – illegal fishing. Rather than picking out faces, the software tracks the movement of fishing boats to root out illegal behavior. And soon, using a twist on facial recognition, it may be able to recognize when a boat's haul includes endangered and protected fish.
The latest weapon in the fight against illegal fishing? Artificial intelligence
Facial recognition software is most commonly known as a tool to help police identify a suspected criminal by using machine learning algorithms to analyze his or her face against a database of thousands or millions of other faces. The larger the database, with a greater variety of facial features, the smarter and more successful the software becomes – effectively learning from its mistakes to improve its accuracy. Now, this type of artificial intelligence is starting to be used in fighting a specific but pervasive type of crime – illegal fishing. Rather than picking out faces, the software tracks the movement of fishing boats to root out illegal behavior. And soon, using a twist on facial recognition, it may be able to recognize when a boat's haul includes endangered and protected fish.