Information Technology
DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning Shan Zhang 2 Yongxin He2 Ting Zhang
Current techniques for detecting AI-generated text are largely confined to manual feature crafting and supervised binary classification paradigms. These methodologies typically lead to performance bottlenecks and unsatisfactory generalizability. Consequently, these methods are often inapplicable for out-of-distribution (OOD) data and newly emerged large language models (LLMs). In this paper, we revisit the task of AI-generated text detection. We argue that the key to accomplishing this task lies in distinguishing writing styles of different authors, rather than simply classifying the text into human-written or AI-generated text.
Bayesian Learning of Sum-Product Networks
Martin Trapp, Robert Peharz, Hong Ge, Franz Pernkopf, Zoubin Ghahramani
Sum-product networks (SPNs) are flexible density estimators and have received significant attention due to their attractive inference properties. While parameter learning in SPNs is well developed, structure learning leaves something to be desired: Even though there is a plethora of SPN structure learners, most of them are somewhat ad-hoc and based on intuition rather than a clear learning principle. In this paper, we introduce a well-principled Bayesian framework for SPN structure learning.
On the Equivalence between Online and Private Learnability beyond Binary Classification
Alon et al. [4] and Bun et al. [10] recently showed that online learnability and private PAC learnability are equivalent in binary classification. We investigate whether this equivalence extends to multi-class classification and regression. First, we show that private learnability implies online learnability in both settings. Our extension involves studying a novel variant of the Littlestone dimension that depends on a tolerance parameter and on an appropriate generalization of the concept of threshold functions beyond binary classification. Second, we show that while online learnability continues to imply private learnability in multi-class classification, current proof techniques encounter significant hurdles in the regression setting. While the equivalence for regression remains open, we provide non-trivial sufficient conditions for an online learnable class to also be privately learnable.
our responses to individual questions and comments. Reviewer # 1 We thank for your positive feedback
We really appreciate the time and expertise you have invested in these reviews. We thank for your positive feedback. It is still possible that there is another method to prove the result for regression. Presentation of Algorithm 2: We will make Algorithm 2 more formal and make the proof of Theorem 8 more readable. A multi-class classification algorithm based on ordinal regression machine.) Thanks for raising this issue, and we will update Section 2.3 to clarify that the Realizable or agnostic settinig: For the sake of clear presentation, we only discussed the realizable setting in the paper.
Model-Based Transfer Learning for Contextual Reinforcement Learning
Deep reinforcement learning (RL) is a powerful approach to complex decision making. However, one issue that limits its practical application is its brittleness, sometimes failing to train in the presence of small changes in the environment. Motivated by the success of zero-shot transfer--where pre-trained models perform well on related tasks--we consider the problem of selecting a good set of training tasks to maximize generalization performance across a range of tasks. Given the high cost of training, it is critical to select training tasks strategically, but not well understood how to do so. We hence introduce Model-Based Transfer Learning (MBTL), which layers on top of existing RL methods to effectively solve contextual RL problems. MBTL models the generalization performance in two parts: 1) the performance set point, modeled using Gaussian processes, and 2) performance loss (generalization gap), modeled as a linear function of contextual similarity. MBTL combines these two pieces of information within a Bayesian optimization (BO) framework to strategically select training tasks. We show theoretically that the method exhibits sublinear regret in the number of training tasks and discuss conditions to further tighten regret bounds.
53fde96fcc4b4ce72d7739202324cd49-AuthorFeedback.pdf
Effect (i) was observed in several papers (e.g. Since our goal is to also achieve (i), we focus on the conditioned models for (ii) in Tab. 4 and use the We have now conducted additional experiments for Tab. 4 using As requested, we have filled some gaps in the tables: For Tab. 1 Yes, all models define a distribution on the same variables. R3: The only baseline is based on [13]. In Tab. 1 above, we also compare Model (3) does capture the correlations of error vectors within each region via the error term ɛ.
Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with the reverse dynamic method (known in the literature as "adjoint method") to train neural ODEs on classification, density estimation, and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method that was confirmed and validated by extensive numerical experiments for several standard benchmarks.