Convolutional Neural Processes for Inpainting Satellite Images: Application to Water Body Segmentation (Papers Track)
Alexander Pondaven (Imperial College London); Mart Bakler (Imperial College London); Donghu Guo (Imperial College London); Hamzah Hashim (Imperial College London); Martin G Ignatov (Imperial college London); Samir Bhatt (Imperial College London); Seth Flaxman (Oxford); Swapnil Mishra (Imperial College London); Elie Alhajjar (USMA); Harrison Zhu (Imperial College London)
Abstract
The widespread availability of satellite images has allowed researchers to monitor the impact of climate on socio-economic and environmental issues through examples like crop and water body classification to measure food scarcity and risk of flooding. However, a common issue of satellite images is missing values due to measurement defects, which render them unusable by existing methods without data imputation. To repair the data, inpainting methods can be employed, which are based on classical PDEs or interpolation methods. Recently, deep learning approaches have shown promise in this realm, however many of these methods do not explicitly take into account the inherent spatio-temporal structure of satellite images. In this work, we cast satellite image inpainting as a meta-learning problem, and implement Convolutional Neural Processes (ConvNPs) in which we frame each satellite image as its own task or 2D regression problem. We show that ConvNPs outperform classical methods and state-of-the-art deep learning inpainting models on a scanline problem for LANDSAT 7 satellite images, assessed on a variety of in- and out-of-distribution images. Our results successfully match the performance of clean images on a downstream water body segmentation task in Canada.