Goto

Collaborating Authors

 Yin, Ruicheng


Explainable Synthetic Image Detection through Diffusion Timestep Ensembling

arXiv.org Artificial Intelligence

Recent advances in diffusion models have enabled the creation of deceptively real images, posing significant security risks when misused. In this study, we reveal that natural and synthetic images exhibit distinct differences in the high-frequency domains of their Fourier power spectra after undergoing iterative noise perturbations through an inverse multi-step denoising process, suggesting that such noise can provide additional discriminative information for identifying synthetic images. Based on this observation, we propose a novel detection method that amplifies these differences by progressively adding noise to the original images across multiple timesteps, and train an ensemble of classifiers on these noised images. To enhance human comprehension, we introduce an explanation generation and refinement module to identify flaws located in AI-generated images. Additionally, we construct two new datasets, GenHard and GenExplain, derived from the GenImage benchmark, providing detection samples of greater difficulty and high-quality rationales for fake images. Extensive experiments show that our method achieves state-of-the-art performance with 98.91% and 95.89% detection accuracy on regular and harder samples, increasing a minimal of 2.51% and 3.46% compared to baselines. Furthermore, our method also generalizes effectively to images generated by other diffusion models. Our code and datasets will be made publicly available.


Searching for Best Practices in Retrieval-Augmented Generation

arXiv.org Artificial Intelligence

Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, particularly in specialized domains. While many RAG approaches have been proposed to enhance large language models through query-dependent retrievals, these approaches still suffer from their complex implementation and prolonged response times. Typically, a RAG workflow involves multiple processing steps, each of which can be executed in various ways. Here, we investigate existing RAG approaches and their potential combinations to identify optimal RAG practices. Through extensive experiments, we suggest several strategies for deploying RAG that balance both performance and efficiency. Moreover, we demonstrate that multimodal retrieval techniques can significantly enhance question-answering capabilities about visual inputs and accelerate the generation of multimodal content using a "retrieval as generation" strategy.


Decoding Continuous Character-based Language from Non-invasive Brain Recordings

arXiv.org Artificial Intelligence

Over the past decade, advancements in brain-computer interfaces have demonstrated the feasibility of decoding various forms of communication, such as speech sounds [80, 81], hand gestures [79, 82], articulatory movements [77, 78], and other signals [76] from intracranial recordings. Despite their efficacy, the requirement for invasive brain surgery limits the applicability of these decoding methods to patients with severe impediments in speech or communication due to neurodegenerative diseases, strokes, or traumatic brain injuries. In contrast, non-invasive recordings, particularly those employing functional magnetic resonance imaging (fMRI) [72, 74], magnetoencephalography (MEG) and electroencephalography (EEG) [73], have demonstrated the ability to record rich linguistic information, and decoding natural language from such non-invasive recordings holds the potential for broader applications in both restorative interventions and augmentative technologies. Previous efforts to decode natural language from non-invasive recordings have primarily focused on recognizing letters, words, or fragments within a predetermined set of possibilities [66-69, 72, 73]. A recent breakthrough has demonstrated the feasibility of decoding continuous language from non-invasive recordings of native English speakers [65].