panopticon
Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation
Waldmann, Leonard, Shah, Ando, Wang, Yi, Lehmann, Nils, Stewart, Adam J., Xiong, Zhitong, Zhu, Xiao Xiang, Bauer, Stefan, Chuang, John
Earth observation (EO) data features diverse sensing platforms with varying spectral bands, spatial resolutions, and sensing modalities. While most prior work has constrained inputs to fixed sensors, a new class of any-sensor foundation models able to process arbitrary sensors has recently emerged. Contributing to this line of work, we propose Panopticon, an any-sensor foundation model built on the DINOv2 framework. We extend DINOv2 by (1) treating images of the same geolocation across sensors as natural augmentations, (2) subsampling channels to diversify spectral input, and (3) adding a cross attention over channels as a flexible patch embedding mechanism. By encoding the wavelength and modes of optical and synthetic aperture radar sensors, respectively, Panopticon can effectively process any combination of arbitrary channels. In extensive evaluations, we achieve state-of-the-art performance on GEO-Bench, especially on the widely-used Sentinel-1 and Sentinel-2 sensors, while out-competing other any-sensor models, as well as domain adapted fixed-sensor models on unique sensor configurations. Panopticon enables immediate generalization to both existing and future satellite platforms, advancing sensor-agnostic EO.
Panopticon: a novel deep learning model to detect single transit events with no prior data filtering in PLATO light curves
Vivien, H. G., Deleuil, M., Jannsen, N., De Ridder, J., Seynaeve, D., Carpine, M. -A., Zerah, Y.
To prepare for the analyses of the future PLATO light curves, we develop a deep learning model, Panopticon, to detect transits in high precision photometric light curves. Since PLATO's main objective is the detection of temperate Earth-size planets around solar-type stars, the code is designed to detect individual transit events. The filtering step, required by conventional detection methods, can affect the transit, which could be an issue for long and shallow transits. To protect transit shape and depth, the code is also designed to work on unfiltered light curves. We trained the model on a set of simulated PLATO light curves in which we injected, at pixel level, either planetary, eclipsing binary, or background eclipsing binary signals. We also include a variety of noises in our data, such as granulation, stellar spots or cosmic rays. The approach is able to recover 90% of our test population, including more than 25% of the Earth-analogs, even in the unfiltered light curves. The model also recovers the transits irrespective of the orbital period, and is able to retrieve transits on a unique event basis. These figures are obtained when accepting a false alarm rate of 1%. When keeping the false alarm rate low (<0.01%), it is still able to recover more than 85% of the transit signals. Any transit deeper than 180ppm is essentially guaranteed to be recovered. This method is able to recover transits on a unique event basis, and does so with a low false alarm rate. Thanks to light curves being one-dimensional, model training is fast, on the order of a few hours per model. This speed in training and inference, coupled to the recovery effectiveness and precision of the model make it an ideal tool to complement, or be used ahead of, classical approaches.
How smart are Gmail's 'smart replies'?
The philosopher Jeremy Bentham was famed for his panopticon, a hypothetical circular prison that was designed in such a way that its inmates never knew whether or not they were being observed. This would, his theory went, encourage prisoners to presume they were always being watched, and thus act accordingly. No true version of the prison was ever really built, and the word itself only now lives on due to its prodigious utility within breathless op-eds about surveillance culture, mostly written by people who've already overused references to Orwell and Kafka. The genius of today's boring dystopia has been to offer this surveillance as a feature, not a bug; to cast that all-seeing-eye not as a malevolent shadowy jailer, but as the world's most boring personal assistant. Nowhere is this truer than with Gmail smart replies, the pocket panopticon that now resides in every inbox.