Deep learning models can accurately map genomic DNA sequences to associated functional molecular readouts such as protein-DNA binding data.

Weaknesses: The most cited methods in this space use a different pooling architecture and train multi-task for dozens of epochs. In contrast, the authors train single task and choose the model achieved after one or two epochs as best. These differences may contribute to the rapid overfitting and saliency variance. Do multi-task models trained for longer improve motif annotation? Does your method also improve motif annotation in that framework?

This paper proposes a novel Fourier-based attribution prior, which can facilitate the application of deep learning to sequence data by improving the interpretation of the learned model. There is significant technical novelty, the authors presented strong experimental results, and the proposed approach is likely of high impact in the field of genomics. Reviewers are largely satisfied with the author feedback. Therefore, the submission is clearly above the bar for the acceptance to NeurIPS.

Deep learning models can accurately map genomic DNA sequences to associated functional molecular readouts such as protein-DNA binding data. "attribution") scores inferred from these models can highlight predictive sequence motifs and syntax. Unfortunately, these models are prone to overfitting and are sensitive to random initializations, often resulting in noisy and irreproducible attributions that obfuscate underlying motifs. To address these shortcomings, we propose a novel attribution prior, where the Fourier transform of input-level attribution scores are computed at training-time, and high-frequency components of the Fourier spectrum are penalized. We evaluate different model architectures with and without our attribution prior, training on genome-wide binary labels or continuous molecular profiles.