shiftnet
Automated transport separation using the neural shifted proper orthogonal decomposition
Zorawski, Beata, Burela, Shubhaditya, Krah, Philipp, Marmin, Arthur, Schneider, Kai
This paper presents a neural network-based methodology for the decomposition of transport-dominated fields using the shifted proper orthogonal decomposition (sPOD). Classical sPOD methods typically require an a priori knowledge of the transport operators to determine the co-moving fields. However, in many real-life problems, such knowledge is difficult or even impossible to obtain, limiting the applicability and benefits of the sPOD. To address this issue, our approach estimates both the transport and co-moving fields simultaneously using neural networks. This is achieved by training two sub-networks dedicated to learning the transports and the co-moving fields, respectively. Applications to synthetic data and a wildland fire model illustrate the capabilities and efficiency of this neural sPOD approach, demonstrating its ability to separate the different fields effectively.
HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery
Deudon, Michel, Kalaitzis, Alfredo, Goytom, Israel, Arefin, Md Rifat, Lin, Zhichao, Sankaran, Kris, Michalski, Vincent, Kahou, Samira E., Cornebise, Julien, Bengio, Yoshua
Generative deep learning has sparked a new wave of Super-Resolution (SR) algorithms that enhance single images with impressive aesthetic results, albeit with imaginary details. Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views. This is important for satellite monitoring of human impact on the planet -- from deforestation, to human rights violations -- that depend on reliable imagery. To this end, we present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion: (i) co-registration, (ii) fusion, (iii) up-sampling, and (iv) registration-at-the-loss. Co-registration of low-resolution views is learned implicitly through a reference-frame channel, with no explicit registration mechanism. We learn a global fusion operator that is applied recursively on an arbitrary number of low-resolution pairs. We introduce a registered loss, by learning to align the SR output to a ground-truth through ShiftNet. We show that by learning deep representations of multiple views, we can super-resolve low-resolution signals and enhance Earth Observation data at scale. Our approach recently topped the European Space Agency's MFSR competition on real-world satellite imagery.