Earth Observation & Monitoring

Tutorials

Blog Posts

Webinars

Workshop Papers

Venue Title
ICLR 2024 Structured spectral reconstruction for scalable soil organic carbon inference (Papers Track)
Abstract and authors: (click to expand)

Abstract: Measuring soil organic carbon (SOC) inexpensively and accurately is crucial for soil health monitoring and agricultural decarbonization. Hyperspectral imaging is commonly evaluated as an inexpensive alternative to dry combustion for SOC measurement, but existing end-to-end approaches trained to predict SOC content from spectral data frequently fail to generalize when applied outside of their ground-truth geographic sampling distributions. Using stratified data from the USDA Rapid Carbon Assessment (RaCA), we demonstrate a method to improve model generalization out-of-distribution by training SOC regression alongside models that reconstruct input spectra. Because hyperspectra can be collected from remote platforms such as drones and satellites, this approach raises the possibility of using large hyperspectral Earth observation datasets to transfer SOC inference models to remote geographies where geographically-dense ground-truth data collection may be expensive or impossible. By replacing the decoder with a simple physics-informed model, we also learn an interpretable spectral signature of SOC, confirming its dark hue and expected reflectance troughs. Finally, we show that catastrophic generalization failures can be better addressed with these architectures by fine-tuning on large quantities of hyperspectral data.

Authors: Evan A Coleman (MIT); Sujay Nair (Georgia Institute of Technology); Xinyi Zeng (Coho Climate Advisors); Elsa Olivetti (MIT Department of Materials Science & Engineering)
ICLR 2024 Grapevine Disease Prediction Using Climate Variables from Multi-Sensor Remote Sensing Imagery via a Transformer Model (Papers Track)
Abstract and authors: (click to expand)

Abstract: Early detection and management of grapevine diseases are important in pursuing sustainable viticulture. This paper introduces a novel framework leveraging the TabPFN model to forecast blockwise grapevine diseases using climate variables from multi-sensor remote sensing imagery. By integrating advanced machine learning techniques with detailed environmental data, our approach significantly enhances the accuracy and efficiency of disease prediction in vineyards. The TabPFN model's experimental evaluations showcase comparable performance to traditional gradient-boosted decision trees, such as XGBoost, CatBoost, and LightGBM. The model's capability to process complex data and provide per-pixel disease-affecting probabilities enables precise, targeted interventions, contributing to more sustainable disease management practices. Our findings underscore the transformative potential of combining Transformer models with remote sensing data in precision agriculture, offering a scalable solution for improving crop health and productivity while reducing environmental impact.

Authors: Weiying Zhao (Deep Planet); Natalia Efremova (Queen Mary University London)
ICLR 2024 Black carbon plumes from gas flaring in North Africa identified from multi-spectral imagery with deep learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Black carbon (BC) is an important pollutant aerosol emitted by numerous human activities, including gas flaring. Improper combustion in flaring activities can release large amounts of BC, which is harmful to human health and has a strong climate warming effect. To our knowledge, no study has ever directly monitored BC emissions from satellite imagery. Previous works quantified BC emissions indirectly, by applying emission coefficients to flaring volumes estimated from satellite imagery. Here, we develop a deep learning framework and apply it to Sentinel-2 imagery over North Africa during 2022 to detect and quantify BC emissions from gas flaring. We find that BC emissions in this region amount to about 1 million tCO2eq, or 1 million passenger cars, more than a quarter of which are due to 10 sites alone. This work demonstrates the operational monitoring of BC emissions from flaring, a key step in implementing effective mitigation policies to reduce the climate impact of oil and gas operations.

Authors: Alexandre Tuel (Galeio); Thomas Kerdreux (INRIA/ ENS); Louis THIRY (ENS Paris)
ICLR 2024 From spectra to biophysical insights: end-to-end learning with a biased radiative transfer model (Papers Track)
Abstract and authors: (click to expand)

Abstract: Advances in machine learning have boosted the use of Earth observation data for climate change research. Yet, the interpretability of machine-learned representations remains a challenge, particularly in understanding forests' biophysical reactions to climate change. Traditional methods in remote sensing that invert radiative transfer models (RTMs) to retrieve biophysical variables from spectral data often fail to account for biases inherent in the RTM, especially for complex forests. We propose to integrate RTMs into an auto-encoder architecture, creating an end-to-end learning approach. Our method not only corrects biases in RTMs but also outperforms traditional techniques for variable retrieval like neural network regression. Furthermore, our framework has potential generally for inverting biased physical models.

Authors: Yihang She (University of Cambridge); Clement Atzberger (Mantle Labs); Andrew Blake (University of Cambridge, Mantle Labs); Srinivasan Keshav (University of Cambridge)
ICLR 2024 A Deep Learning Technology Suite for Cost-Effective Sequestered CO2 Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon capture and storage (CCS) is a way of reducing carbon emissions to help tackle global warming. Injecting CO2 into rock formations and preventing it from escaping to the surface is a main step in a CCS project. Therefore, monitoring of geologically sequestered CO2 is important for CCS security assessment. Time-lapse seismic (4D seismic) is one of the most effective tools for CO2 monitoring. Unfortunately, the main challenge of 4D seismic is the high cost due to repeated monitoring seismic data acquisition surveys and the subsequent time-consuming data processing that involves imaging and inversion. To address this, we developed a technology suite powered by deep learning engines that significantly reduces the cost by (1) acquiring very sparse monitoring data; (2) firing multiple seismic sources simultaneously; (3) converting 2D images to 3D volume; (4) enforcing repeatability between baseline data and monitoring data; and (5) nonlinearly mapping seismic data to subsurface property model to bypass complex wave-equation-based seismic data processing procedures.

Authors: Wenyi Hu (SLB); Son Phan (SLB); Cen Li (SLB); Aria Abubakar (SLB)
ICLR 2024 Global High Resolution CO2 monitoring using Super Resolution (Papers Track)
Abstract and authors: (click to expand)

Abstract: Monitoring Greenhouse Gases (GHGs) concentrations and emissions is essential to mitigate climate change. Thanks to the large amount of satellite data available, it is now possible to understand GHGs' behaviours at a broad scale. However, due to remote sensing devices technological limitations, the task of global high resolution (HR) monitoring remains an open problem. To avoid waiting for new missions and better data to be generated, it is therefore relevant to experiment with processing methods able to improve existing datasets. Our paper proposes to apply Super Resolution (SR), a Deep Learning (DL) approach commonly used in Computer Vision (CV), on global L3 satellite data. We produce a daily high resolution global CO2 dataset that opens the door for globally consistent point source monitoring.

Authors: Andrianirina Rakotoharisoa (Imperial College London); Rossella Arcucci (Imperial College London)
ICLR 2024 Machine Learning for the Detection of Arctic Melt Ponds from Infrared Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Melt ponds are pools of water on Arctic summer sea ice that play an important role in the Arctic climate system. Retrieving their coverage is essential to better understand and predict the rapidly changing Arctic, but current data are limited. The goal of this study is to enhance melt pond data by developing a method that segments thermal infrared (TIR) airborne imagery into melt pond, sea ice, and ocean classes. Due to temporally and spatially varying surface temperatures, we use a data-driven deep learning approach to solve this task. We adapt and fine-tune AutoSAM, a Segment Anything-based segmentation model. We make the code, data, and models available online.

Authors: Marlena Reil (University of Osnabrück and University of Bremen, Institute of Environmental Physics); Gunnar Spreen (University of Bremen, Institute of Environmental Physics); Marcus Huntemann (University of Bremen, Institute of Environmental Physics); Lena Buth (Alfred Wegener Institute); Dennis Wilson (University of Toulouse, ISAE-Supaero)
ICLR 2024 SkyImageNet: Towards a large-scale sky image dataset for solar power forecasting (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The variability of solar photovoltaic (PV) output, particularly that caused by rapidly changing cloud dynamics, challenges the reliability of renewable energy systems. Solar forecasting based on cloud observations collected by ground-level sky cameras shows promising performance in anticipating short-term solar power fluctuations. However, current deep learning methods often rely on a single dataset with limited sample diversity for training, and thus generalize poorly to new locations and different sky conditions. Moreover, the lack of a standardized dataset hinders the consistent comparison of existing solar forecasting methods. To close these gaps, we propose to build a large-scale standardized sky image dataset --- SkyImageNet --- by assembling, harmonizing, and processing suitable open-source datasets collected in various geographical locations. An accompanying python package will be developed to streamline the process of utilizing SkyImageNet in a machine learning framework. We hope that the outcomes of this project will foster the development of more robust forecasting systems, advance the comparability of short-term solar forecasting model performances, and further facilitate the transition to the next generation of sustainable energy systems.

Authors: Yuhao Nie (Massachusetts Institute of Technology); Quentin Paletta (European Space Research Institute); Sherrie Wang (MIT)
ICLR 2024 A Benchmark Dataset for Meteorological Downscaling (Proposals Track)
Abstract and authors: (click to expand)

Abstract: High spatial resolution in atmospheric representations is crucial across Earth science domains, but global reanalysis datasets like ERA5 often lack the detail to capture local phenomena due to their coarse resolution. Recent efforts have leveraged deep neural networks from computer vision to enhance the spatial resolution of meteorological data, showing promise for statistical downscaling. However, methodological diversity and insufficient comparisons with traditional downscaling techniques challenge these advancements. Our study introduces a benchmark dataset for statistical downscaling, utilizing ERA5 and the finer-resolution COSMO-REA6, to facilitate direct comparisons of downscaling methods for 2m temperature, global (solar) irradiance and 100m wind fields. Accompanying U-Net, GAN, and transformer models with a suite of evaluation metrics aim to standardize assessments and promote transparency and confidence in applying deep learning to meteorological downscaling.

Authors: Michael Langguth (Juelich Supercomputing Centre - Forschungszentrum Juelich); Paula Harder (Mila); Irene Schicker (Geos); Ankit Patnala (Juelich Supercomputing Centre - Forschungszentrum Juelich); Sebastian Lehner (GeoSphere Austria); Konrad Mayer (GeoSphere Austria); Markus Dabernig (GeoSphere Austria)
ICLR 2024 One Prompt Fits All: Visual Prompt-Tuning for Remote Sensing Segmentation (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Image segmentation is crucial in climate change research for analyzing satellite imagery. This technique is vital for ecosystems mapping, natural disasters assessment, and urban and agricultural planning. The advent of vision-based foundational models like the Segment Anything Model (SAM) opens new avenues in climate research and remote sensing (RS). SAM can perform segmentation tasks on any object from manually-crafted prompts. However, the efficacy of SAM largely depends on the quality of these prompts. This issue is particularly pronounced with RS data, which are inherently complex. To use SAM for accurate segmentation at scale for RS, one would need to create complex prompts for each image, which typically involves selecting dozens of points. To address this, we introduce Prompt-Tuned SAM (PT-SAM), a method that minimizes the need for manual input through a trainable, lightweight prompt embedding. This embedding captures key semantic information for specific objects of interest that would be applicable to unseen images. Our approach merges the zero-shot generalization capabilities of the pre-trained SAM model with supervised learning. Importantly, the training process for the prompt embedding not only has minimal hardware requirements, allowing it to be conducted on a CPU, but it also requires only a small dataset. With PT-SAM, image segmentation on RS data can be performed at scale without human intervention, achieving accuracies comparable to those of human-designed prompts with SAM. For example, PT-SAM can be used for analyzing forest cover across vast areas, a key factor in understanding the impact of human activities on forests. Its capability to segment a multitude of images makes it ideal for monitoring widespread land-cover changes, providing deeper insights into urbanization. This tutorial will explore how to train and utilize PT-SAM for large-scale segmentation tasks, specifically focusing on training embeddings that capture forests, and buildings.

Authors: Xuekun Wang (Vector Institute); John Willes (Vector Institute); Deval Pandya (Vector Institute)
NeurIPS 2023 EarthPT: a foundation model for Earth Observation (Papers Track)
Abstract and authors: (click to expand)

Abstract: We introduce EarthPT -- an Earth Observation (EO) pretrained transformer. EarthPT is a 700 million parameter decoding transformer foundation model trained in an autoregressive self-supervised manner and developed specifically with EO use-cases in mind. We demonstrate that EarthPT is an effective forecaster that can accurately predict future pixel-level surface reflectances across the 400-2300 nm range well into the future. For example, forecasts of the evolution of the Normalised Difference Vegetation Index (NDVI) have a typical error of approximately 0.05 (over a natural range of -1 -> 1) at the pixel level over a five month test set horizon, out-performing simple phase-folded models based on historical averaging. We also demonstrate that embeddings learnt by EarthPT hold semantically meaningful information and could be exploited for downstream tasks such as highly granular, dynamic land use classification. Excitingly, we note that the abundance of EO data provides us with -- in theory -- quadrillions of training tokens. Therefore, if we assume that EarthPT follows neural scaling laws akin to those derived for Large Language Models (LLMs), there is currently no data-imposed limit to scaling EarthPT and other similar ‘Large Observation Models.’

Authors: Michael J Smith (Aspia Space); Luke Fleming (Aspia Space); James Geach (Aspia Space)
NeurIPS 2023 Artificial Intelligence for Methane Mitigation : Through an Automated Determination of Oil and Gas Methane Emissions Profiles (Papers Track)
Abstract and authors: (click to expand)

Abstract: The oil and gas sector is the second largest anthropogenic emitter of methane, which is responsible for approximately 25% of global warming since pre-industrial times. In order to mitigate methane atmospheric emissions from oil and gas industry, the potential emitting infrastructure must be monitored. Initiatives such as the Methane Alert and Response System (MARS), launched by the United Nations Environment Program, aim to locate significant emissions events, alert relevant stakeholders, as well as monitor and track progress in mitigation efforts. To achieve this goal, an automated solution is needed for consistent monitoring across multiple oil and gas basins around the world. Most methane emissions analysis studies propose post-emission analysis. The works and future guidelines presented in this paper aim to provide an automated collection of informed methane emissions by oil and gas site and infrastructure which are necessary to dress emission profile in near real time. This proposed framework also permits to create action margins to reduce methane emissions by passing from post methane emissions analysis to forecasting methods.

Authors: Jade Eva Guisiano (Sorbonne / ISEP / Polytechnique / UNEP); Thomas LAUVAUX (Université de Reims); Eric Moulines (Ecole Polytechnique); Jérémie Sublime (ISEP)
NeurIPS 2023 Weakly-semi-supervised object detection in remotely sensed imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Deep learning for detecting objects in remotely sensed imagery can enable new technologies for important applications including mitigating climate change. However, these models often require large datasets labeled with bounding box annotations which are expensive to curate, prohibiting the development of models for new tasks and geographies. To address this challenge, we develop weakly-semi-supervised object detection (WSSOD) models on remotely sensed imagery which can leverage a small amount of bounding boxes together with a large amount of point labels that are easy to acquire at scale in geospatial data. We train WSSOD models which use large amounts of point-labeled images with varying fractions of bounding box labeled images in FAIR1M and a wind turbine detection dataset, and demonstrate that they substantially outperform fully supervised models trained with the same amount of bounding box labeled images on both datasets. Furthermore, we find that the WSSOD models trained with 2-10x fewer bounding box labeled images can perform similarly to or outperform fully supervised models trained on the full set of bounding-box labeled images. We believe that the approach can be extended to other remote sensing tasks to reduce reliance on bounding box labels and increase development of models for impactful applications.

Authors: Ji Hun Wang (Stanford University); Jeremy Irvin (Stanford University); Beri Kohen Behar (Stanford University); Ha Tran (Stanford University); Raghav Samavedam (Stanford University); Quentin Hsu (Stanford University); Andrew Ng (Stanford University)
NeurIPS 2023 Prototype-oriented Unsupervised Change Detection for Disaster Management (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change has led to an increased frequency of natural disasters such as floods and cyclones. This emphasizes the importance of effective disaster monitoring. In response, the remote sensing community has explored change detection methods. These methods are primarily categorized into supervised techniques, which yield precise results but come with high labeling costs, and unsupervised techniques, which eliminate the need for labeling but involve intricate hyperparameter tuning. To address these challenges, we propose a novel unsupervised change detection method named Prototype-oriented Unsupervised Change Detection for Disaster Management (PUCD). PUCD captures changes by comparing features from pre-event, post-event, and prototype-oriented change synthesis images via a foundational model, and refines results using the Segment Anything Model (SAM). Although PUCD is an unsupervised change detection, it does not require complex hyperparameter tuning. We evaluate PUCD framework on the LEVIR-Extension dataset and the disaster dataset and it achieves state-of-the-art performance compared to other methods on the LEVIR-Extension dataset.

Authors: YoungTack Oh (SI Analytics); Minseok Seo (si-analytics); Doyi Kim (SI Analytics); Junghoon Seo (SI Analytics)
NeurIPS 2023 Flowering Onset Detection: Traditional Learning vs. Deep Learning Performance in a Sparse Label Context (Papers Track)
Abstract and authors: (click to expand)

Abstract: Detecting temporal shifts in plant flowering times is of increasing importance in a context of climate change, with applications in plant ecology, but also health, agriculture, and ecosystem management. However, scaling up plant-level monitoring is cost prohibitive, and flowering transitions are complex and difficult to model. We develop two sets of approaches to detect the onset of flowering at large-scale and high-resolution. Using fine grain temperature data with domain knowledge based features and traditional machine learning models provides the best performance. Using satellite data, with deep learning to deal with high dimensionality and transfer learning to overcome ground truth label sparsity, is a challenging but promising approach, as it reaches good performance with more systematically available data.

Authors: Mauricio Soroco (University of British Columbia); Joel Hempel (University of British Columbia); Xinze Xiong (University of British Columbia); Mathias Lécuyer (University of British Columbia); Joséphine Gantois (University of British Columbia)
NeurIPS 2023 Glacier Movement Prediction with Attention-based Recurrent Neural Networks and Satellite Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Studying glacier movements is crucial because of their indications for global climate change and its effects on local land masses. Building on established methods for glacier movement prediction from Landsat-8 satellite imaging data, we develop an attention-based deep learning model for time series data prediction of glacier movements. In our approach, the Normalized Difference Snow Index is calculated from the Landsat-8 spectral reflectance bands for data of the Parvati Glacier (India) to quantify snow and ice in the scene images, which is then used for time series prediction. Based on this data, a newly developed Long-Short Term Memory Encoder-decoder neural network model is trained, incorporating a Multi-head Self Attention mechanism in the decoder. The model shows promising results, making the prediction of optical flow vectors from pure model predictions possible.

Authors: Jonas Müller (University of Tübingen); Raphael Braun (University of Tübingen); Hendrik P. A. Lensch (University of Tübingen); Nicole Ludwig (University of Tübingen)
NeurIPS 2023 Detailed Glacier Area Change Analysis in the European Alps with Deep Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Glacier retreat is a key indicator of climate change and requires regular updates of the glacier area. Recently, the release of a new inventory for the European Alps showed that glaciers continued to retreat at about 1.3% per year from 2003 to 2015. The outlines were produced by manually correcting the results of a semi-automatic method applied to Sentinel-2 imagery. In this work we develop a fully-automatic pipeline based on Deep Learning to investigate the evolution of the glaciers in the Alps from 2015 to present (2023). After outlier filtering, we provide individual estimates for around 1300 glaciers, representing 87% of the glacierized area. Regionally we estimate an area loss of -1.8% per year, with large variations between glaciers. Code and data are available at https://github.com/dcodrut/glacier_mapping_alps_tccml.

Authors: Codrut-Andrei Diaconu (DLR); Jonathan Bamber (Technical University of Munich)
NeurIPS 2023 Segment-then-Classify: Few-shot instance segmentation for environmental remote sensing (Papers Track)
Abstract and authors: (click to expand)

Abstract: Instance segmentation is pivotal for environmental sciences and climate change research, facilitating important tasks from land cover classification to glacier monitoring. This paper addresses the prevailing challenges associated with data scarcity when using traditional models like YOLOv8 by introducing a novel, data-efficient workflow for instance segmentation. The proposed Segment-then-Classify (STC) strategy leverages the zero-shot capabilities of the novel Segment Anything Model (SAM) to segment all objects in an image and then uses a simple classifier such as the Vision Transformer (ViT) to identify objects of interest thereafter. Evaluated on the VHR-10 dataset, our approach demonstrated convergence with merely 40 examples per class. YOLOv8 requires 3 times as much data to achieve the STC's peak performance. The highest performing class in the VHR-10 dataset achieved a near-perfect mAP@0.5 of 0.99 using the STC strategy. However, performance varied greatly across other classes due to the SAM model’s occasional inability to recognize all relevant objects, indicating a need for refining the zero-shot segmentation step. The STC workflow therefore holds promise for advancing few-shot learning for instance segmentation in environmental science.

Authors: Yang Hu (University of California, Santa Barbara); Kelly Caylor (UCSB); Anna S Boser (UCSB)
NeurIPS 2023 Lightweight, Pre-trained Transformers for Remote Sensing Timeseries (Papers Track)
Abstract and authors: (click to expand)

Abstract: Machine learning models for parsing remote sensing data have a wide range of societally relevant applications, but labels used to train these models can be difficult or impossible to acquire. This challenge has spurred research into self-supervised learning for remote sensing data. Current self-supervised learning approaches for remote sensing data draw significant inspiration from techniques applied to natural images. However, remote sensing data has important differences from natural images -- for example, the temporal dimension is critical for many tasks and data is collected from many complementary sensors. We show we can create significantly smaller performant models by designing architectures and self-supervised training techniques specifically for remote sensing data. We introduce the Pretrained Remote Sensing Transformer (Presto), a transformer-based model pre-trained on remote sensing pixel-timeseries data. Presto excels at a wide variety of globally distributed remote sensing tasks and performs competitively with much larger models while requiring far less compute. Presto can be used for transfer learning or as a feature extractor for simple models, enabling efficient deployment at scale.

Authors: Gabriel Tseng (NASA Harvest); Ruben Cartuyvels (KULeuven); Ivan Zvonkov (University of Maryland); Mirali Purohit (Arizona State University (ASU)); David Rolnick (McGill University, Mila); Hannah R Kerner (Arizona State University)
NeurIPS 2023 Top-down Green-ups: Satellite Sensing and Deep Models to Predict Buffelgrass Phenology (Papers Track)
Abstract and authors: (click to expand)

Abstract: An invasive species of grass known as "buffelgrass" contributes to severe wildfires and biodiversity loss in the Southwest United States. We tackle the problem of predicting buffelgrass "green-ups" (i.e. readiness for herbicidal treatment). To make our predictions, we explore temporal, visual and multi-modal models that combine satellite sensing and deep learning. We find that all of our neural-based approaches improve over conventional buffelgrass green-up models, and discuss how neural model deployment promises significant resource savings.

Authors: Lucas Rosenblatt (NYU); Bin Han (University of Washington); Erin Posthumus (USA NPN); Theresa Crimmins (USA NPN); Bill G Howe (University of Washington)
NeurIPS 2023 Data Assimilation using ERA5, ASOS, and the U-STN model for Weather Forecasting over the UK (Papers Track)
Abstract and authors: (click to expand)

Abstract: In recent years, the convergence of data-driven machine learning models with Data Assimilation (DA) offers a promising avenue for enhancing weather forecasting. This study delves into this emerging trend, presenting our methodologies and outcomes. We harnessed the UK's local ERA5 850 hPa temperature data and refined the U-STN12 global weather forecasting model, tailoring its predictions to the UK's climate nuances. From the ASOS network, we sourced t2m data, representing ground observations across the UK. We employed the advanced kriging method with a polynomial drift term for consistent spatial resolution. Furthermore, Gaussian noise was superimposed on the ERA5 T850 data, setting the stage for ensuing multi-time step virtual observations. Probing into the assimilation impacts, the ASOS t2m data was integrated with the ERA5 T850 dataset. Our insights reveal that while global forecast models can adapt to specific regions, incorporating atmospheric data in DA significantly bolsters model accuracy. Conversely, the direct assimilation of surface temperature data tends to mitigate this enhancement, tempering the model's predictive prowess.

Authors: WENQI WANG (Imperial College London); Jacob Bieker (Open Climate Fix); Rossella Arcucci (Imperial College London); Cesar Quilodran-Casas (Imperial College London)
NeurIPS 2023 Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data (Papers Track) Overall Best Paper
Abstract and authors: (click to expand)

Abstract: Satellite-based remote sensing is instrumental in the monitoring and mitigation of the effects of anthropogenic climate change. Large scale, high resolution data derived from these sensors can be used to inform intervention and policy decision making, but the timeliness and accuracy of these interventions is limited by use of optical data, which cannot operate at night and is affected by adverse weather conditions. Synthetic Aperture Radar (SAR) offers a robust alternative to optical data, but its associated complexities limit the scope of labelled data generation for traditional deep learning. In this work, we apply a self-supervised pretraining scheme, masked autoencoding, to SAR amplitude data covering 8.7\% of the Earth's land surface area, and tune the pretrained weights on two downstream tasks crucial to monitoring climate change - vegetation cover prediction and land cover classification. We show that the use of this pretraining scheme reduces labelling requirements for the downstream tasks by more than an order of magnitude, and that this pretraining generalises geographically, with the performance gain increasing when tuned downstream on regions outside the pretraining set. Our findings significantly advance climate change mitigation by facilitating the development of task and region-specific SAR models, allowing local communities and organizations to deploy tailored solutions for rapid, accurate monitoring of climate change effects.

Authors: Matthew J Allen (University of Cambridge); Francisco Dorr (Independent); Joseph A Gallego (National University Of Colombia); Laura Martínez-Ferrer (University of Valencia); Freddie Kalaitzis (University of Oxford); Raul Ramos-Pollan (Universidad de Antioquia); Anna Jungbluth (European Space Agency)
NeurIPS 2023 Methane Plume Detection with U-Net Segmentation on Sentinel-2 Image Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Methane emissions have a significant impact on increasing global warming. Satellite-based methane detection methods can help mitigate methane emissions, as they provide a constant and global detection. The Sentinel-2 constellation, in particular, offers frequent and publicly accessible images on a global scale. We propose a deep learning approach to detect methane plumes from Sentinel-2 images. We construct a dataset of 5200 satellite images with identified methane plumes, on which we train a U-Net model. Preliminary results demonstrate that the model is able to correctly identify methane plumes on training data, although generalization to new methane plumes remains challenging. All code, data, and models are made available online.

Authors: Berenice du Baret (ISAE-Supaero); Simon Finos (ISAE-Supaero); Hugo Guiglion (ISAE-Supaero); Dennis Wilson (ISAE)
NeurIPS 2023 Improving Flood Insights: Diffusion-based SAR to EO Image Translation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Driven by the climate crisis, the frequency and intensity of flood events are on the rise. Electro-optical (EO) satellite imagery is commonly used for rapid disaster response. However, its utility in flood situations is limited by cloud cover and during nighttime. An alternative method for flood detection involves using Synthetic Aperture Radar (SAR) data. Despite SAR's advantages over EO in these situations, it has a significant drawback: human analysts often struggle to interpret SAR data. This paper proposes a novel framework, Diffusion-based SAR-to-EO Image Translation (DSE). The DSE framework converts SAR images into EO-like imagery, thereby enhancing their interpretability for human analysis. Experimental results on the Sen1Floods11 and SEN12-FLOOD datasets confirm that the DSE framework provides enhanced visual information and improves performance in all flood segmentation tests.

Authors: Minseok Seo (si-analytics); YoungTack Oh (SI Analytics); Doyi Kim (SI Analytics); Dongmin Kang (SIA); Yeji Choi (SI Analytics)
NeurIPS 2023 Deep Glacier Image Velocimetry: Mapping glacier velocities from Sentinel-2 imagery with deep learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Glacier systems are highly sensitive to climate change and play a pivotal role in global mean sea level rise. As such, it is important to monitor how glacier velocities and ice dynamics evolve under a changing climate. The growing wealth of satellite observations has facilitated the inference of glacier velocities from remote sensing imagery through feature tracking algorithms. At present, these rely on sparse cross-correlation estimates as well as computationally expensive optical flow solutions. Here we present a novel use of deep-learning for estimating annual glacier velocities, utilizing the recurrent optical-flow based architecture, RAFT, on consecutive pairs of optical Sentinel-2 imagery. Our results highlight that deep learning can generate dense per-pixel velocity estimates within an automated framework that utilizes Sentinel-2 images over the French Alps.

Authors: James B Tlhomole (Imperial College London); Matthew Piggott (Imperial College London); Graham Hughes (Imperial College London)
NeurIPS 2023 Simulating the Air Quality Impact of Prescribed Fires Using a Graph Neural Network-Based PM2.5 Emissions Forecasting System (Papers Track)
Abstract and authors: (click to expand)

Abstract: The increasing size and severity of wildfires across western North America have generated dangerous levels of PM2.5 pollution in recent years. In a warming climate, expanding the use of prescribed fires is widely considered to be the most robust fire mitigation strategy. However, reliably forecasting the potential air quality impact from these prescribed fires, a critical ingredient in determining the fires’ location and time, at hourly to daily time scales remains a challenging problem. This paper proposes a novel integration of prescribed fire simulation with a spatio-temporal graph neural network-based PM2.5 forecasting model. The experiments in this work focus on determining the optimal time for implementing prescribed fires in California as well as quantifying the potential air quality trade-offs involved in conducting more prescribed fires outside the fire season.

Authors: Kyleen Liao (Saratoga High School); Jatan Buch (Columbia University); Kara D. Lamb (Columbia University); Pierre Gentine (Columbia University)
NeurIPS 2023 Hyperspectral shadow removal with iterative logistic regression and latent Parametric Linear Combination of Gaussians (Papers Track)
Abstract and authors: (click to expand)

Abstract: Shadow detection and removal is a challenging problem in the analysis of hyperspectral images. Yet, this step is crucial for analyzing data for remote sensing applications like methane detection. In this work, we develop a shadow detection and removal method only based on the spectrum of each pixel and the overall distribution of spectral values. We first introduce Iterative Logistic Regression(ILR) to learn a spectral basis in which shadows can be linearly classified. We then model the joint distribution of the mean radiance and the projection coefficients of the spectra onto the above basis as a parametric linear combination of Gaussians. We can then extract the maximum likelihood mixing parameter of the Gaussians to estimate the shadow coverage and to correct the shadowed spectra. Our correction scheme reduces correction artefacts at shadow borders. The shadow detection and removal method is applied to hyperspectral images from MethaneAIR, a precursor to the satellite MethaneSAT.

Authors: Core Francisco Park (Harvard University); Maya Nasr (Harvard University); Manuel Pérez-Carrasco (University of Concepcion); Eleanor Walker (Harvard University); Douglas Finkbeiner (Harvard University); Cecilia Garraffo (AstroAI at the Center for Astrophysics, Harvard & Smitnsonian)
NeurIPS 2023 Elucidating the Relationship Between Climate Change and Poverty using Graph Neural Networks, Ensemble Models, and Remote Sensing Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate and poverty are intrinsically related: regions with extreme temperatures, large temperature variability, and recurring extreme weather events tend to be ranked among the poorest and most vulnerable to climate change. Nevertheless, there currently is no established method to directly estimate the impact of specific climate variables on poverty and to identify geographical regions at high risk of being negatively affected by climate change. In this work, we propose a new approach based on Graph Neural Networks (GNNs) to estimate the effect of climate and remote sensing variables on poverty indicators measuring Education, Health, Living Standards, and Income. Furthermore, we use the trained models and perturbation analyses to identify the geographical regions most vulnerable to the potential variations in climate variables.

Authors: Parinthapat Pengpun (Bangkok Christian International School); Alessandro Salatiello (University of Tuebingen)
NeurIPS 2023 Sustainability AI copilot: Analyze & ideate at scale to enable positive impact (Papers Track)
Abstract and authors: (click to expand)

Abstract: With the advances in large scale Foundation Models, web scale access to sustainability data, planetary scale satellite data, the opportunity for larger section of the world population to create positive climate impact can be activated by empowering everyone to ideate via AI copilots. The challenge is: How to enable more people to think & take action on climate & Sustainable Development goals?. We develop AI co-pilots to engage broader community for enabling impact at scale by democratizing climate thinking & ideation tools. We demonstrated how ideating with SAI transforms any seed idea into a holistic one, given the relation between climate & social economic aspects. SAI employs Language Models to represent the voice of the often neglected vulnerable people to the brainstorming discussion for inclusive climate action. We demonstrated how SAI can even create another AI that learns geospatial insights and offers advice to prevent humanitarian disasters from climate change. In this work, we conceptualized, designed, implemented & demonstrated Sustainability AI copilot (SAI) & innovated 4 use cases:- SAI enables sustainability enthusiasts to convert early stage budding thoughts into a robust holistic idea by creatively employing a chain of Large Language Models to think with six-thinking hats ideation. SAI can enables non-experts to become geospatial analysts by generating code to analyze planetary scale satellite data. SAI also ideates in multi-modal latent space to explore climate friendly product designs. SAI also enables human right activists to create awareness about inclusion of vulnerable and persons with disability in the climate conversation. SAI even creates AI apps for persons with disability. We demonstrated working prototypes at the project website, https://sites.google.com/view/climate-copilot . Thus, SAI co-pilot empowers everyone to come together to ideate to make progress on climate and related sustainable development goals.

Authors: Rajagopal A (Indian Institute of Technology); Nirmala V (Queen Marys); Immanuel Raja (Karunya University); Arun V (NIT)
NeurIPS 2023 Assessing data limitations in ML-based LCLU (Proposals Track)
Abstract and authors: (click to expand)

Abstract: This study addresses the accuracy challenge in Global Land Use and Land Cover (LULC) maps, crucial for policy making towards climate change mitigation. We evaluate two LULC products based on advanced machine learning techniques across two representative nations, Ecuador and Germany, employing a novel accuracy metric. The analysis unveils a notable accuracy enhancement in the convolutional neural network-based approach against the random forest model used for comparison. Our findings emphasize the potential of sophisticated machine learning methodologies in advancing LULC mapping accuracy, an essential stride towards data-driven, climate-relevant land management and policy decisions.

Authors: Angel Encalada-Davila (ESPOL); Christian Tutiven (ESPOL University); Jose E Cordova-Garcia (ESPOL)
NeurIPS 2023 Sand Mining Watch: Leveraging Earth Observation Foundation Models to Inform Sustainable Development (Proposals Track)
Abstract and authors: (click to expand)

Abstract: As the major ingredient of concrete and asphalt, sand is vital to economic growth, and will play a key role in aiding the transition to a low carbon society. However, excessive and unregulated sand mining in the Global South has high socio-economic and environmental costs, and amplifies the effects of climate change. Sand mines are characterized by informality and high temporal variability, and data on the location and extent of these mines tends to be sparse. We propose to build custom sand-mine detection tools by fine-tuning foundation models for earth observation, which leverage self supervised learning - a cost-effective and powerful approach in sparse data regimes. Our preliminary results show that these methods outperform fully supervised approaches, with the best performing model achieving an average precision score of 0.57 for this challenging task. These tools allow for real-time monitoring of sand mining activity and can enable more effective policy and regulation, to inform sustainable development.

Authors: Ando Shah (UC Berkeley); Suraj R Nair (UC Berkeley); Tom Boehnel (TU Munich); Joshua Blumenstock (University of California, Berkeley)
NeurIPS 2023 Aquaculture Mapping: Detecting and Classifying Aquaculture Ponds using Deep Learning (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Mapping aquaculture ponds is critical for restoration, conservation, and climate adaptation efforts. Aquaculture can contribute to high levels of water pollution from untreated effluent and negatively impact coastal ecosystems. Large-scale aquaculture is also a significant driver in mangrove deforestation, thus reducing the world’s carbon sinks and exacerbating the effects of climate change. However, finding and mapping these ponds on the ground can be highly labor and time-intensive. Most existing automated techniques are focused only on spatial location and do not consider production intensification, which is also crucial to understanding their impact on the surrounding ecosystem. We can classify them into two main types: a) Extensive ponds, which are large, irregularly-shaped ponds that rely on natural productivity, and b) intensive ponds which are smaller and regularly shaped. Intensive ponds use machinery such as aerators that maximize production and also result in the characteristic presence of air bubbles on the pond’s surface. The features of these two types of ponds make them distinguishable and detectable from satellite imagery. In this tutorial, we will discuss types of aquaculture ponds in detail and demonstrate how they can be detected and classified using satellite imagery. The tutorial will introduce an open dataset of human-labeled aquaculture ponds in the Philippines and Indonesia. Using this dataset, the tutorial will use semantic segmentation to map out similar ponds over an entire country and classify them as either extensive or intensive, going through the entire process of i) satellite imagery retrieval, ii) preprocessing these images into a training-ready dataset, iii) model training, and iv) finally model rollout on a sample area. Throughout, the tutorial will leverage PyTorch Lightning, a machine learning framework that provides a simplified and streamlined interface for model experimentation and deployment. This tutorial aims to discuss the relevance of aquaculture ponds in climate adaptation and equip users with the necessary inputs and tools to perform their own ML-powered earth observation projects.

Authors: John Christian G Nacpil (Thinking Machines Data Science, Inc.); Joshua Cortez (Thinking Machines Data Science)
ICLR 2023 Mitigating climate and health impact of small-scale kiln industry using multi-spectral classifier and deep learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Industrial air pollution has a direct health impact and is a major contributor to climate change. Small scale industries particularly bull-trench brick kilns are one of the major causes of air pollution in South Asia often creating hazardous levels of smog that is injurious to human health. To mitigate the climate and health impact of the kiln industry, fine-grained kiln localization at different geographic locations is needed. Kiln localization using multi-spectral remote sensing data such as vegetation index results in a noisy estimates whereas use of high-resolution imagery is infeasible due to cost and compute complexities. This paper proposes a fusion of spatio-temporal multi-spectral data with high-resolution imagery for detection of brick kilns within the "Brick-Kiln-Belt" of South Asia. We first perform classification using low-resolution spatio-temporal multi-spectral data from Sentinel-2 imagery by combining vegetation, burn, build up and moisture indices. Then orientation aware object detector: YOLOv3 (with theta value) is implemented for removal of false detections and fine-grained localization. Our proposed technique, when compared with other benchmarks, results in a 21 times improvement in speed with comparable or higher accuracy when tested over multiple countries.

Authors: Usman Nazir (Lahore University of Management Sciences); Murtaza Taj (Lahore University of Management Sciences); Momin Uppal (Lahore University of Management Sciences); Sara khalid (University of Oxford)
ICLR 2023 Coregistration of Satellite Image Time Series Through Alignment of Road Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Due to climate change, thawing permafrost affects transportation infrastructure in northern regions. Tracking deformations over time of these structures can allow identifying the most vulnerable sections to permafrost degradation and implement climate adaptation strategies. The Sentinel-2 mission provides data well-suited for multitemporal analysis due to its high temporal resolution and multispectral coverage. However, the geometrical misalignment of Sentinel-2 imagery makes this analysis challenging. Towards the goal of estimating the deformation of linear infrastructure in northern Canada, we propose an automatic subpixel coregistration algorithm for satellite image time series based on the matching of binary masks of roads produced by a deep learning model. We demonstrate the feasibility of achieving subpixel coregistration through alignment of roads on a small dataset of high-resolution Sentinel-2 images from the region of Gillam in northern Canada. This is the first step towards training a road deformation prediction model.

Authors: Andres Felipe Perez Murcia (University of Manitoba); Pooneh Maghoul (University of Manitoba); Ahmed Ashraf (University of Manitoba)
ICLR 2023 Improving a Shoreline Forecasting Model with Symbolic Regression (Papers Track)
Abstract and authors: (click to expand)

Abstract: Given the current context of climate change and the increasing population densities at coastal zones around the globe, there is an increasing need to be able to predict the development of our coasts. Recent advances in artificial intelligence allow for automatic analysis of observational data. Symbolic Regression (SR) is a type of Machine Learning algorithm that aims to find interpretable symbolic expressions that can explain relations in the data. In this work, we aim to study the problem of forecasting shoreline change using SR. We make use of Cartesian Genetic Programming (CGP) in order to encode and improve upon ShoreFor, a physical shoreline prediction model. During training, CGP individuals are evaluated and selected according to their predictive score at five different coastal sites. This work presents a comparison between a CGP-evolved model and the base ShoreFor model. In addition to evolution's ability to produce well-performing models, it demonstrates the usefulness of SR as a research tool to gain insight into the behaviors of shorelines in various geographical zones.

Authors: Mahmoud AL NAJAR (Laboratory of Spatial Geophysics and Oceanography Studies); Rafael ALMAR (Laboratory of Spatial Geophysics and Oceanography Studies); Erwin BERGSMA (CNES); Jean-Marc DELVIT (CNES); Dennis Wilson (ISAE)
ICLR 2023 A simplified machine learning based wildfire ignition model from insurance perspective (Papers Track)
Abstract and authors: (click to expand)

Abstract: In the context of climate change, wildfires are becoming more frequent, intense, and prolonged in the western US, particularly in California. Wildfires cause catastrophic socio-economic losses and are projected to worsen in the near future. Inaccurate estimates of fire risk put further pressure on wildfire (re)insurance and cause many homes to lose wildfire insurance coverage. Efficient and effective prediction of fire ignition is one step towards better fire risk assessment. Here we present a simplified machine learning-based fire ignition model at yearly scale that is well suited to the use case of one-year term wildfire (re)insurance. Our model yields a recall, precision, and the area under the precision-recall curve of 0.69, 0.86 and 0.81, respectively, for California, and significantly higher values of 0.82, 0.90 and 0.90, respectively, for the populated area, indicating its good performance. In addition, our model feature analysis reveals that power line density, enhanced vegetation index (EVI), vegetation optical depth (VOD), and distance to the wildland-urban interface stand out as the most important features determining ignitions. The framework of this simplified ignition model could easily be applied to other regions or genesis of other perils like hurricane, and it paves the road to a broader and more affordable safety net for homeowners.

Authors: Yaling Liu (OurKettle Inc); Son Le (OurKettle Inc.); Yufei Zou (Our Kettle, Inc.); mojtaba Sadgedhi (OurKettle Inc.); Yang Chen (University of California, Irvine); Niels Andela (BeZero Carbon); Pierre Gentine (Columbia University)
ICLR 2023 Disentangling observation biases to monitor spatio-temporal shifts in species distributions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The accelerated pace of environmental change due to anthropogenic activities makes it more important than ever to understand current and future ecosystem dynamics at a global scale. Species observations stemming from citizen science platforms are increasingly leveraged to gather information about the geographic distributions of many species. However, their usability is limited by the strong biases inherent to these community-driven efforts. These biases in the sampling effort are often treated as noise that has to be compensated for. In this project, we posit that better modelling the sampling effort (including the usage of the different platforms across countries, local accessibility, attractiveness of the location for platform users, affinity of different user groups for different species, etc.) is the key towards improving Species Distribution Models (SDM) using observations from citizen science platforms, thus opening up the possibility of leveraging them to monitor changes in species distributions and population densities.

Authors: Diego Marcos (Inria); Christophe Botella (); Ilan Havinga (Wageningen University); Dino Ienco (INRAE); Cassio F. Dantas (TETIS, INRAE, Univ Montpellier); Pierre Alliez (INRIA Sophie-Antipolis, France); Alexis Joly (INRIA, FR)
ICLR 2023 Bayesian Inference of Severe Hail in Australia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Severe hailstorms are responsible for some of the most costly insured weather events in Australia and can cause significant damage to homes, businesses, and agriculture. However their response to climate change remains uncertain, in large part due to the challenges of observing severe hailstorms. We propose a novel Bayesian approach which explicitly models known biases and uncertainties of current hail observations to produce more realistic estimates of severe hail risk from existing observations. Training this model on data from south-east Queensland, Australia, suggests that previous analyses of severe hail that did not account for this uncertainty may produce poorly calibrated risk estimates. Preliminary evaluation on withheld data confirms that our model produces well-calibrated probabilities and is applicable out of sample. Whilst developed for hail, we highlight also the generality of our model and its potential applications to other severe weather phenomena and areas of climate change adaptation and mitigation.

Authors: Isabelle C Greco (University of New South Wales); Steven Sherwood (University of New South Wales); Timothy Raupach (University of New South Wales); Gab Abramowitz (University of New South Wales)
ICLR 2023 Understanding forest resilience to drought with Shapley values (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Increases in drought frequency, intensity, and duration due to climate change are threatening forests around the world. Climate-driven tree mortality is associated with devastating ecological and societal consequences, including the loss of carbon sequestration, habitat provisioning, and water filtration services. A spatially fine-grained understanding of the site characteristics making forests more resilient to drought is still lacking. Furthermore, the complexity of drought effects on forests, which can be cumulative and delayed, demands investigation of the most appropriate drought indices. In this study, we aim to gain a better understanding of the temporal and spatial drivers of drought-induced changes in forest vitality using Shapley values, which allow for the relevance of predictors to be quantified locally. A better understanding of the contribution of meteorological and environmental factors to trees’ response to drought can support forest managers aiming to make forests more climate-resilient.

Authors: Stenka Vulova (Technische Universität Berlin); Alby Duarte Rocha (Technische Universität Berlin); Akpona Okujeni (Humboldt-Universität zu Berlin); Johannes Vogel (Freie Universität Berlin); Michael Förster (Technische Universität Berlin); Patrick Hostert (Humboldt-Universität zu Berlin); Birgit Kleinschmit (Technische Universität Berlin)
ICLR 2023 EfficientTempNet: Temporal Super-Resolution of Radar Rainfall (Papers Track)
Abstract and authors: (click to expand)

Abstract: Rainfall data collected by various remote sensing instruments such as radars or satellites has different space-time resolutions. This study aims to improve the temporal resolution of radar rainfall products to help with more accurate climate change modeling and studies. In this direction, we introduce a solution based on EfficientNetV2, namely EfficientTempNet, to increase the temporal resolution of radar-based rainfall products from 10 minutes to 5 minutes. We tested EfficientRainNet over a dataset for the state of Iowa, US, and compared its performance to three different baselines to show that EfficientTempNet presents a viable option for better climate change monitoring.

Authors: Bekir Z Demiray (University of Iowa); Muhammed A Sit (The University of Iowa); Ibrahim Demir (University of Iowa)
ICLR 2023 Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Running climate simulations informs us of future climate change. However, it is computationally expensive to resolve complex climate processes numerically. As one way to speed up climate simulations, neural networks have been used to downscale climate variables from fast-running low-resolution simulations. So far, all neural network downscaling models can only downscale input samples with a pre-defined upsampling factor. In this work, we propose a Fourier neural operator downscaling model. It trains with data of a small upsampling factor and then can zero-shot downscale its input to arbitrary unseen high-resolutions. Evaluated on Navier-Stokes equation solution data and ERA5 water content data, our downscaling model demonstrates better performance than widely used convolutional and adversarial generative super-resolution models in both learned and zero-shot downscaling. Our model's performance is further boosted when a constraint layer is applied. In the end, we show that by combining our downscaling model with a low-resolution numerical PDE solver, the downscaled solution outperforms the solution of the state-of-the-art high-resolution data-driven solver. Our model can be used to cheaply and accurately generate arbitrarily high-resolution climate simulation data with fast-running low-resolution simulation as input.

Authors: Qidong Yang (New York University); Paula Harder (Fraunhofer ITWM); Venkatesh Ramesh (University of Montreal, Mila); Alex Hernandez-Garcia (Mila - Quebec AI Institute); Daniela Szwarcman (IBM Research); Prasanna Sattigeri (IBM Research); Campbell D Watson (IBM Reserch); David Rolnick (McGill University, Mila)
NeurIPS 2022 Attention-Based Scattering Network for Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Multi-channel satellite imagery, from stacked spectral bands or spatiotemporal data, have meaningful representations for various atmospheric properties. Combining these features in an effective manner to create a performant and trustworthy model is of utmost importance to forecasters. Neural networks show promise, yet suffer from unintuitive computations, fusion of high-level features, and may be limited by the quantity of available data. In this work, we leverage the scattering transform to extract high-level features without additional trainable parameters and introduce a separation scheme to bring attention to independent input channels. Experiments show promising results on estimating tropical cyclone intensity and predicting the occurrence of lightning from satellite imagery.

Authors: Jason Stock (Colorado State University); Charles Anderson (Colorado State University)
NeurIPS 2022 Discovering Interpretable Structural Model Errors in Climate Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Inaccuracies in the models of the Earth system, i.e., structural and parametric model errors, lead to inaccurate climate change projections. Errors in the model can originate from unresolved phenomena due to a low numerical resolution, as well as misrepresentations of physical phenomena or boundaries (e.g., orography). Therefore, such models lead to inaccurate short--term forecasts of weather and extreme events, and more importantly, long term climate projections. While calibration methods have been introduced to address for parametric uncertainties, e.g., by better estimation of system parameters from observations, addressing structural uncertainties, especially in an interpretable manner, remains a major challenge. Therefore, with increases in both the amount and frequency of observations of the Earth system, algorithmic innovations are required to identify interpretable representations of the model errors from observations. We introduce a flexible, general-purpose framework to discover interpretable model errors, and show its performance on a canonical prototype of geophysical turbulence, the two--level quasi--geostrophic system. Accordingly, a Bayesian sparsity--promoting regression framework is proposed, that uses a library of kernels for discovery of model errors. As calculating the library from noisy and sparse data (e.g., from observations) using convectional techniques leads to interpolation errors, here we use a coordinate-based multi--layer embedding to impute the sparse observations. We demonstrate the importance of alleviating spectral bias, and propose a random Fourier feature layer to reduce it in the proposed embeddings, and subsequently enable an accurate discovery. Our framework is demonstrated to successfully identify structural model errors due to linear and nonlinear processes (e.g., radiation, surface friction, advection), as well as misrepresented orography.

Authors: Rambod Mojgani (Rice University); Ashesh K Chattopadhyay (Rice University); Pedram Hassanzadeh (Rice University)
NeurIPS 2022 Scene-to-Patch Earth Observation: Multiple Instance Learning for Land Cover Classification (Papers Track)
Abstract and authors: (click to expand)

Abstract: Land cover classification (LCC), and monitoring how land use changes over time, is an important process in climate change mitigation and adaptation. Existing approaches that use machine learning with Earth observation data for LCC rely on fully-annotated and segmented datasets. Creating these datasets requires a large amount of effort, and a lack of suitable datasets has become an obstacle in scaling the use of LCC. In this study, we propose Scene-to-Patch models: an alternative LCC approach utilising Multiple Instance Learning (MIL) that requires only high-level scene labels. This enables much faster development of new datasets whilst still providing segmentation through patch-level predictions, ultimately increasing the accessibility of using LCC for different scenarios. On the DeepGlobe-LCC dataset, our approach outperforms non-MIL baselines on both scene- and patch-level prediction. This work provides the foundation for expanding the use of LCC in climate change mitigation methods for technology, government, and academia.

Authors: Joseph Early (University of Southampton); Ying-Jung C Deweese (Georgia Insititute of Technology); Christine Evers (University of Southampton); Sarvapali Ramchurn (University of Southampton)
NeurIPS 2022 Detecting Methane Plumes using PRISMA: Deep Learning Model and Data Augmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: The new generation of hyperspectral imagers, such as PRISMA, has improved significantly our detection capability of methane (CH4) plumes from space at high spatial resolution (∼30m). We present here a complete framework to identify CH4 plumes using images from the PRISMA satellite mission and a deep learning technique able to automatically detect plumes over large areas. To compensate for the sparse database of PRISMA images, we trained our model by transposing high resolution plumes from Sentinel-2 to PRISMA. Our methodology avoids computationally expensive synthetic plume from Large Eddy Simulations while generating a broad and realistic training database, and paves the way for large-scale detection of methane plumes using future hyperspectral sensors (EnMAP, EMIT, CarbonMapper).

Authors: Alexis Groshenry (Kayrros); Clément Giron (Kayrros); Alexandre d'Aspremont (CNRS, DI, Ecole Normale Supérieure; Kayrros); Thomas Lauvaux (University of Reims Champagne Ardenne, GSMA, UMR 7331); Thibaud Ehret (Centre Borelli)
NeurIPS 2022 Bridging the Microwave Data Gap; Using Bayesian Deep Learning to “See” the Unseen (Papers Track)
Abstract and authors: (click to expand)

Abstract: Having microwave data with the spatial and temporal resolution of infrared data would provide a large positive impact on many climate and weather applications. We demonstrate that Bayesian deep learning is a promising technique for both creating and improving synthetic microwave data from infrared data. We report 0.7% mean absolute percentage error for 183+/-3 GHz microwave brightness temperature and uncertainty metrics and find that more training data is needed to achieve improved performance at 166 GHz, 37 GHz, and 23 GHz. Analysis of the spatial distribution of uncertainty reveals that additional cloud data will provide the greatest increase in skill, which will potentially allow for generation of many secondary products derived from microwave data in the future.

Authors: Pedro Ortiz (Naval Postgraduate School); Eleanor Casas (Naval Postgraduate School); Marko Orescanin (Naval Postgraduate School); Scott Powell (Naval Postgraduate School)
NeurIPS 2022 Learning evapotranspiration dataset corrections from water cycle closure supervision (Papers Track)
Abstract and authors: (click to expand)

Abstract: Evapotranspiration (ET) is one of the most uncertain components of the global water cycle. Improving global ET estimates is needed to better our understanding of the global water cycle so as to forecast the consequences of climate change on the future of global water resource distribution. This work presents a methodology to derive monthly corrections of global ET datasets at 0.25 degree resolution. We use ML to generalize sparse catchment-level water cycle closure residual information to global and dense pixel-level residuals. Our model takes a probabilistic view on ET datasets and their correction that we use to regress catchment-level residuals using a sum-aggregated supervision. Using four global ET datasets, we show that our learned model has learned ET corrections that accurately generalize its water cycle-closure results to unseen catchments.

Authors: Tristan E.M Hascoet (Kobe University); Victor Pellet (LERMA); Filipe Aires (LERMA)
NeurIPS 2022 Convolutional Neural Processes for Inpainting Satellite Images: Application to Water Body Segmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: The widespread availability of satellite images has allowed researchers to monitor the impact of climate on socio-economic and environmental issues through examples like crop and water body classification to measure food scarcity and risk of flooding. However, a common issue of satellite images is missing values due to measurement defects, which render them unusable by existing methods without data imputation. To repair the data, inpainting methods can be employed, which are based on classical PDEs or interpolation methods. Recently, deep learning approaches have shown promise in this realm, however many of these methods do not explicitly take into account the inherent spatio-temporal structure of satellite images. In this work, we cast satellite image inpainting as a meta-learning problem, and implement Convolutional Neural Processes (ConvNPs) in which we frame each satellite image as its own task or 2D regression problem. We show that ConvNPs outperform classical methods and state-of-the-art deep learning inpainting models on a scanline problem for LANDSAT 7 satellite images, assessed on a variety of in- and out-of-distribution images. Our results successfully match the performance of clean images on a downstream water body segmentation task in Canada.

Authors: Alexander Pondaven (Imperial College London); Mart Bakler (Imperial College London); Donghu Guo (Imperial College London); Hamzah Hashim (Imperial College London); Martin G Ignatov (Imperial college London); Samir Bhatt (Imperial College London); Seth Flaxman (Oxford); Swapnil Mishra (Imperial College London); Elie Alhajjar (USMA); Harrison Zhu (Imperial College London)
NeurIPS 2022 Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Satellite image analysis has important implications for land use, urbanization, and ecosystem monitoring. Deep learning methods can facilitate the analysis of different satellite modalities, such as electro-optical (EO) and synthetic aperture radar (SAR) imagery, by supporting knowledge transfer between the modalities to compensate for individual shortcomings. Recent progress has shown how distributional alignment of neural network embeddings can produce powerful transfer learning models by employing a sliced Wasserstein distance (SWD) loss. We analyze how this method can be applied to Sentinel-1 and -2 satellite imagery and develop several extensions toward making it effective in practice. In an application to few-shot Local Climate Zone (LCZ) prediction, we show that these networks outperform multiple common baselines on datasets with a large number of classes. Further, we provide evidence that instance normalization can significantly stabilize the training process and that explicitly shaping the embedding space using supervised contrastive learning can lead to improved performance.

Authors: Marcel Hussing (University of Pennsylvania); Karen Li (University of Pennsylvania); Eric Eaton (University of Pennsylvania)
NeurIPS 2022 Exploring Randomly Wired Neural Networks for Climate Model Emulation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts, but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this manuscript, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them to their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired ones and assess the impact on model performance for models with 1 million and 10 million parameters. We find average performance improvements of 4.2% across model complexities and prediction tasks, with substantial performance improvements of up to 16.4% in some cases. Furthermore, we find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.

Authors: William J Yik (Harvey Mudd College); Sam J Silva (The University of Southern California); Andrew Geiss (Pacific Northwest National Laboratory); Duncan Watson-Parris (University of Oxford)
NeurIPS 2022 Remote estimation of geologic composition using interferometric synthetic-aperture radar in California’s Central Valley (Papers Track)
Abstract and authors: (click to expand)

Abstract: California's Central Valley is the national agricultural center, producing 1/4 of the nation’s food. However, land in the Central Valley is sinking at a rapid rate (as much as 20 cm per year) due to continued groundwater pumping. Land subsidence has a significant impact on infrastructure resilience and groundwater sustainability. In this study, we aim to identify specific regions with different temporal dynamics of land displacement and find relationships with underlying geological composition. Then, we aim to remotely estimate geologic composition using interferometric synthetic aperture radar (InSAR)-based land deformation temporal changes using machine learning techniques. We identified regions with different temporal characteristics of land displacement in that some areas (e.g., Helm) with coarser grain geologic compositions exhibited potentially reversible land deformation (elastic land compaction). We found a significant correlation between InSAR-based land deformation and geologic composition using random forest and deep neural network regression models. We also achieved significant accuracy with 1/4 sparse sampling to reduce any spatial correlations among data, suggesting that the model has the potential to be generalized to other regions for indirect estimation of geologic composition. Our results indicate that geologic composition can be estimated using InSAR-based land deformation data. In-situ measurements of geologic composition can be expensive and time consuming and may be impractical in some areas. The generalizability of the model sheds light on high spatial resolution geologic composition estimation utilizing existing measurements.

Authors: Kyongsik Yun (California Institute of Technology); Kyra Adams (California Institute of Technology); John Reager (California Institute of Technology); Zhen Liu (California Institute of Technology); Caitlyn Chavez (California Institute of Technology); Michael Turmon (California Institute of Technology); Thomas Lu (California Institute of Technology)
NeurIPS 2022 Cross Modal Distillation for Flood Extent Mapping (Papers Track)
Abstract and authors: (click to expand)

Abstract: The increasing intensity and frequency of floods is one of the many consequences of our changing climate. In this work, we explore ML techniques that improve the flood detection module of an operational early flood warning system. Our method exploits an unlabelled dataset of paired multi-spectral and Synthetic Aperture Radar (SAR) imagery to reduce the labeling requirements of a purely supervised learning method. Past attempts have used such unlabelled data by creating weak labels out of them, but end up learning the label mistakes in those weak labels. Motivated by knowledge distillation and semi supervised learning, we explore the use of a teacher to train a student with the help of a small hand labeled dataset and a large unlabelled dataset. Unlike the conventional self distillation setup, we propose a cross modal distillation framework that transfers supervision from a teacher trained on richer modality (multi-spectral images) to a student model trained on SAR imagery. The trained models are then tested on the Sen1Floods11 dataset. Our model outperforms the Sen1Floods11 SAR baselines by an absolute margin of 4.15% pixel wise Intersection-over-Union (IoU) on the test split.

Authors: Shubhika Garg (Google); Ben Feinstein (Google); Shahar Timnat (Google); Vishal V Batchu (Google); gideon dror (The Academic College of Tel-Aviv-Yaffo); Adi Gerzi Rosenthal (Google); Varun Gulshan (Google Research)
NeurIPS 2022 Identifying Compound Climate Drivers of Forest Mortality with β-VAE (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to lead to higher rates of forest mortality. Forest mortality is a complex phenomenon driven by the interaction of multiple climatic variables at multiple temporal scales, further modulated by the current state of the forest (e.g. age, stem diameter, and leaf area index). Identifying the compound climate drivers of forest mortality would greatly improve understanding and projections of future forest mortality risk. Observation data are, however, limited in accuracy and sample size, particularly in regard to forest state variables and mortality events. In contrast, simulations with state-of-the-art forest models enable the exploration of novel machine learning techniques for associating forest mortality with driving climate conditions. Here we simulate 160,000 years of beech, pine and spruce forest dynamics with the forest model FORMIND. We then apply β-VAE to learn disentangled latent representations of weather conditions and identify those that are most likely to cause high forest mortality. The learned model successfully identifies three characteristic climate representations that can be interpreted as different compound drivers of forest mortality.

Authors: Mohit Anand (Helmholtz Centre for Environmental Research - UFZ); Lily-belle Sweet (Helmholtz Centre for Environmental Research - UFZ); Gustau Camps-Valls (Universitat de València); Jakob Zscheischler (Helmholtz Centre for Environmental Research - UFZ)
NeurIPS 2022 Deep Learning for Rapid Landslide Detection using Synthetic Aperture Radar (SAR) Datacubes (Papers Track)
Abstract and authors: (click to expand)

Abstract: With climate change predicted to increase the likelihood of landslide events, there is a growing need for rapid landslide detection technologies that help inform emergency responses. Synthetic Aperture Radar (SAR) is a remote sensing technique that can provide measurements of affected areas independent from weather or lighting conditions. Usage of SAR, however, is hindered by domain knowledge that is necessary for the pre-processing steps and its interpretation requires expert knowledge. We provide simplified, pre-processed, machine-learning ready SAR datacubes for four globally located landslide events obtained from several Sentinel-1 satellite passes before and after a landslide triggering event together with segmentation maps of the landslides. From this dataset, using the Hokkaido, Japan datacube, we study the feasibility of SAR-based landslide detection with supervised deep learning (DL). Our results demonstrate that DL models can be used to detect landslides from SAR data, achieving an Area under the Precision-Recall curve exceeding 0.7. We find that additional satellite visits enhance detection performance, but that early detection is possible when SAR data is combined with terrain information from a digital elevation model. This can be especially useful for time-critical emergency interventions.

Authors: Vanessa Boehm (UC Berkeley); Wei Ji Leong (The Ohio State University); Ragini Bal Mahesh (German Aerospace Center DLR); Ioannis Prapas (National Observatory of Athens); Siddha Ganju (Nvidia); Freddie Kalaitzis (University of Oxford); Edoardo Nemni (United Nations Satellite Centre (UNOSAT)); Raul Ramos-Pollan (Universidad de Antioquia)
NeurIPS 2022 Deep Learning for Global Wildfire Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to aggravate wildfire activity through the exacerbation of fire weather. Improving our capabilities to anticipate wildfires on a global scale is of uttermost importance for mitigating their negative effects. In this work, we create a global fire dataset and demonstrate a prototype for predicting the presence of global burned areas on a sub-seasonal scale with the use of segmentation deep learning models. Particularly, we present an open-access global analysis-ready datacube, which contains a variety of variables related to the seasonal and sub-seasonal fire drivers (climate, vegetation, oceanic indices, human-related variables), as well as the historical burned areas and wildfire emissions for 2001-2021. We train a deep learning model, which treats global wildfire forecasting as an image segmentation task and skillfully predicts the presence of burned areas 8, 16, 32 and 64 days ahead of time. Our work motivates the use of deep learning for global burned area forecasting and paves the way towards improved anticipation of global wildfire patterns.

Authors: Ioannis Prapas (National Observatory of Athens); Akanksha Ahuja (NOA); Spyros Kondylatos (National Observatory of Athens); Ilektra Karasante (National Observatory of Athens); Lazaro Alonso (Max Planck Institute for Biogeochemistry); Eleanna Panagiotou (Harokopio University of Athens); Charalampos Davalas (Harokopio University of Athens); Dimitrios Michail (Harokopio University of Athens); Nuno Carvalhais (Max Planck Institute for Biogeochemistry); Ioannis Papoutsis (National Observatory of Athens)
NeurIPS 2022 Positional Encoder Graph Neural Networks for Geographic Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Modeling spatial dependencies in geographic data is of crucial importance for the modeling of our planet. Graph neural networks (GNNs) provide a powerful and scalable solution for modeling continuous spatial data. However, in the absence of further context on the geometric structure of the data, they often rely on Euclidean distances to construct the input graphs. This assumption can be improbable in many real-world settings, where the spatial structure is more complex and explicitly non-Euclidean (e.g., road networks). In this paper, we propose PE-GNN, a new framework that incorporates spatial context and correlation explicitly into the models. Building on recent advances in geospatial auxiliary task learning and semantic spatial embeddings, our proposed method (1) learns a context-aware vector encoding of the geographic coordinates and (2) predicts spatial autocorrelation in the data in parallel with the main task. We show the effectiveness of our approach on two climate-relevant regression tasks: 3d spatial interpolation and air temperature prediction. The code for this study can be accessed via: https://bit.ly/3xDpfyV.

Authors: Konstantin Klemmer (Microsoft Research); Nathan S Safir (University of Georgia); Daniel B Neill (New York University)
NeurIPS 2022 Towards Global Crop Maps with Transfer Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: The continuous increase in global population and the impact of climate change on crop production are expected to affect the food sector significantly. In this context, there is need for timely, large-scale and precise mapping of crops for evidence-based decision making. A key enabler towards this direction are new satellite missions that freely offer big remote sensing data of high spatio-temporal resolution and global coverage. During the previous decade and because of this surge of big Earth observations, deep learning methods have dominated the remote sensing and crop mapping literature. Nevertheless, deep learning models require large amounts of annotated data that are scarce and hard-to-acquire. To address this problem, transfer learning methods can be used to exploit available annotations and enable crop mapping for other regions, crop types and years of inspection. In this work, we have developed and trained a deep learning model for paddy rice detection in South Korea using Sentinel-1 VH time-series. We then fine-tune the model for i) paddy rice detection in France and Spain and ii) barley detection in the Netherlands. Additionally, we propose a modification in the pre-trained weights in order to incorporate extra input features (Sentinel-1 VV). Our approach shows excellent performance when transferring in different areas for the same crop type and rather promising results when transferring in a different area and crop type.

Authors: Hyun-Woo Jo (Korea University); Alkiviadis Marios Koukos (National Observatory of Athens); Vasileios Sitokonstantinou (National Observatory of Athens); Woo-Kyun Lee (Korea University); Charalampos Kontoes (National Observatory of Athens)
NeurIPS 2022 Pyrocast: a Machine Learning Pipeline to Forecast Pyrocumulonimbus (PyroCb) Clouds (Papers Track)
Abstract and authors: (click to expand)

Abstract: Pyrocumulonimbus (pyroCb) clouds are storm clouds generated by extreme wildfires. PyroCbs are associated with unpredictable, and therefore dangerous, wildfire spread. They can also inject smoke particles and trace gases into the upper troposphere and lower stratosphere, affecting the Earth's climate. As global temperatures increase, these previously rare events are becoming more common. Being able to predict which fires are likely to generate pyroCb is therefore key to climate adaptation in wildfire-prone areas. This paper introduces Pyrocast, a pipeline for pyroCb analysis and forecasting. The pipeline's first two components, a pyroCb database and a pyroCb forecast model, are presented. The database brings together geostationary imagery and environmental data for over 148 pyroCb events across North America, Australia, and Russia between 2018 and 2022. Random Forests, Convolutional Neural Networks (CNNs), and CNNs pretrained with Auto-Encoders were tested to predict the generation of pyroCb for a given fire six hours in advance. The best model predicted pyroCb with an AUC of 0.90±0.04.

Authors: Kenza Tazi (University of Cambridge); Emiliano Díaz Salas-Porras (University of Valencia); Ashwin Braude (Institut Pierre-Simon Laplace); Daniel Okoh (National Space Research and Development Agency); Kara D. Lamb (Columbia University); Duncan Watson-Parris (University of Oxford); Paula Harder (Fraunhofer ITWM); Nis Meinert (Pasteur Labs)
NeurIPS 2022 Evaluating Digital Tools for Sustainable Agriculture using Causal Inference (Papers Track)
Abstract and authors: (click to expand)

Abstract: In contrast to the rapid digitalization of several industries, agriculture suffers from low adoption of climate-smart farming tools. Even though AI-driven digital agriculture can offer high-performing predictive functionalities, they lack tangible quantitative evidence on their benefits to the farmers. Field experiments can derive such evidence, but are often costly and time consuming. To this end, we propose an observational causal inference framework for the empirical evaluation of the impact of digital tools on target farm performance indicators. This way, we can increase farmers' trust via enhancing the transparency of the digital agriculture market, and in turn accelerate the adoption of technologies that aim to increase productivity and secure a sustainable and resilient agriculture against a changing climate. As a case study, we perform an empirical evaluation of a recommendation system for optimal cotton sowing, which was used by a farmers' cooperative during the growing season of 2021. We leverage agricultural knowledge to develop the causal graph of the farm system, we use the back-door criterion to identify the impact of recommendations on the yield and subsequently we estimate it using several methods on observational data. The results showed that a field sown according to our recommendations enjoyed a significant increase in yield 12% to 17%.

Authors: Ilias Tsoumas (National Observatory of Athens); Georgios Giannarakis (National Observatory of Athens); Vasileios Sitokonstantinou (National Observatory of Athens); Alkiviadis Marios Koukos (National Observatory of Athens); Dimitra A Loka (Hellenic Agricultural Organization ELGO DIMITRA); Nikolaos S Bartsotas (National Observatory of Athens); Charalampos Kontoes (National Observatory of Athens); Ioannis N Athanasiadis (Wageningen University and Research)
NeurIPS 2022 Nowformer : A Locally Enhanced Temporal Learner for Precipitation Nowcasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: The precipitation video datasets have distinctive meteorological patterns where a mass of fluid moves in a particular direction across the entire frames, and each local area of the fluid has an individual life cycle from initiation to maturation to decay. This paper proposes a novel transformer-based model for precipitation nowcasting that can extract global and local dynamics within meteorological characteristics. The experimental results show our model achieves state-of-the-art performances on the precipitation nowcasting benchmark.

Authors: Jinyoung Park (KAIST); Inyoung Lee (KAIST); Minseok Son (KAIST); Seungju Cho (KAIST); Changick Kim (KAIST)
NeurIPS 2022 Using uncertainty-aware machine learning models to study aerosol-cloud interactions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions (ACI) include various effects that result from aerosols entering a cloud, and affecting cloud properties. In general, an increase in aerosol concentration results in smaller droplet sizes which leads to larger, brighter, longer-lasting clouds that reflect more sunlight and cool the Earth. The strength of the effect is however heterogeneous, meaning it depends on the surrounding environment, making ACI one of the most uncertain effects in our current climate models. In our work, we use causal machine learning to estimate ACI from satellite observations by reframing the problem as a treatment (aerosol) and outcome (change in droplet radius). We predict the causal effect of aerosol on clouds with uncertainty bounds depending on the unknown factors that may be influencing the impact of aerosol. Of the three climate models evaluated, we find that only one plausibly recreates the trend, lending more credence to its estimate cooling due to ACI.

Authors: Maëlys Solal (University of Oxford); Andrew Jesson (University of Oxford); Yarin Gal (University of Oxford); Alyson Douglas (University of Oxford)
NeurIPS 2022 Performance evaluation of deep segmentation models on Landsat-8 imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Contrails, short for condensation trails, are line-shaped ice clouds produced by aircraft engine exhaust when they fly through the cold and humid air. They generate a greenhouse effect by absorbing or directing back to Earth approximately 33% of emitted outgoing longwave radiation. They account for over half of the climate change resulting from aviation activities. Avoiding contrails and adjusting flight routes could be an inexpensive and effective way to reduce their impact. An accurate, automated, and reliable detection algorithm is required to develop and evaluate contrail avoidance strategies. Advancement in contrail detection has been severely limited due to several factors, primarily due to a lack of quality-labelled data. Recently, McCloskey et al. proposed a large human-labelled Landsat-8. Each contrail is carefully labelled with various inputs in various scenes of Landsat-8 satellite imagery. In this work, we benchmark several popular segmentation models with combinations of different loss functions and encoder backbones. This work is the first to apply state-of-the-art segmentation techniques to detect contrails in low-orbit satellite imagery. Our work can also be used as an open benchmark for contrail segmentation.

Authors: Akshat Bhandari (Manipal Institute of Technology, Manipal); Pratinav Seth (Manipal Institute of Technology); Sriya Rallabandi (Manipal Institute of Technology); Aditya Kasliwal (Manipal Institute of Technology); Sanchit Singhal (Manipal Institute of Technology)
NeurIPS 2022 Guided Transformer Network for Detecting Methane Emissions in Sentinel-2 Satellite Imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane (CH4) is the chief contributor to global climate change and its mitigation is targeted by the EU, US and jurisdictions worldwide [2]. Recent studies have shown that imagery from the multi-spectral instrument on Sentinel-2 satellites is capable of detecting and estimating large methane emissions. However, most of the current methods rely on temporal relations between a ratio of shortwave-infrared spectra and assume relatively constant ground conditions, and availability of ground information on when there was no methane emission on site. To address such limitations we propose a guided query-based transformer neural network architecture, that will detect and quantify methane emissions without dependence on temporal information. The guided query aspect of our architecture is driven by a Sentinel Enhanced Matched Filter (SEMF) approach, also discussed in this paper. Our network uses all 12 spectral channels of Sentinel-2 imagery to estimate ground terrain and detect methane emissions. No dependence on temporal data makes it more robust to changing ground and terrain conditions and more computationally efficient as it reduces the need to process historical time-series imagery to compute a single date emissions analysis.

Authors: Satish Kumar (University of California, Santa Barbara); William Kingwill (Orbio Earth); Rozanne Mouton (Orbio Earth); Wojciech Adamczyk (ETH, Zurich); Robert Huppertz (Orbio Earth); Evan D Sherwin (Stanford University, Energy and Resources Engineering)
NeurIPS 2022 Deep-S2SWind: A data-driven approach for improving Sub-seasonal wind predictions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: A major transformation to mitigate climate change implies a rapid decarbonisation of the energy system and thus, increasing the use of renewable energy sources, such as wind power. However, renewable resources are strongly dependent on local and large-scale weather conditions, which might be influenced by climate change. Weather-related risk assessments are essential for the energy sector, in particular, for power system management decisions, for which forecasts of climatic conditions from several weeks to months (i.e. sub-seasonal scales) are of key importance. Here, we propose a data-driven approach to predict wind speed at longer lead-times that can benefit the energy sector. The main goal of this study is to assess the potential of machine learning algorithms to predict periods of low wind speed conditions that have a strong impact on the energy sector.

Authors: Noelia Otero Felipe (University of Bern); Pascal Horton (University of Bern)
NeurIPS 2022 An Inversion Algorithm of Ice Thickness and InSAR Data for the State of Friction at the Base of the Greenland Ice Sheet (Proposals Track)
Abstract and authors: (click to expand)

Abstract: With the advent of climate change and global warming, the Greenland Ice Sheet (GrIS) has been melting at an alarming rate, losing over 215 Gt per yr, and accounting for 10% of mean global sea level rise since the 1990s. It is imperative to understand what dynamics are causing ice loss and influencing ice flow in order to successfully project mass changes of ice sheets and associated sea level rise. This work applies machine learning, ice thickness data, and horizontal ice velocity measurements from satellite radar data to quantify the magnitudes and distributions of the basal traction forces that are holding the GrIS back from flowing into the ocean. Our approach uses a hybrid model: InSAR velocity data trains a linear regression model, and these model coefficients are fed into a geophysical algorithm to estimate basal tractions that capture relationships between the ice motion and physical variables. Results indicate promising model performance and reveal significant presence of large basal traction forces around the coastline of the GrIS.

Authors: Aryan Jain (Amador Valley High School); Jeonghyeop Kim (Stony Brook University); William Holt (Stony Brook University)
NeurIPS 2022 Surrogate Modeling for Methane Dispersion Simulations Using Fourier Neural Operator (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane leak detection and remediation are critical for tackling climate change, where methane dispersion simulations play an important role in emission source attribution. As 3D modeling of methane dispersion is often costly and time-consuming, we train a deep-learning-based surrogate model using the Fourier Neural Operator to learn the PDE solver in our study. Our preliminary result shows that our surrogate modeling provides a fast, accurate and cost-effective solution to methane dispersion simulations, thus reducing the cycle time of methane leak detection.

Authors: Qie Zhang (Microsoft); Mirco Milletari (Microsoft); Yagna Deepika Oruganti (Microsoft); Philipp A Witte (Microsoft)
NeurIPS 2022 Forecasting Global Drought Severity and Duration Using Deep Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Drought detection and prediction is challenging due to the slow onset of the event and varying degrees of dependence on numerous physical and socio-economic factors that differentiate droughts from other natural disasters. In this work, we propose DeepXD (Deep learning for Droughts), a deep learning model with 26 physics-informed input features for SPI (Standardised Precipitation Index) forecasting to identify and classify droughts using monthly oceanic indices, global meteorological and vegetation data, location (latitude, longitude) and land cover for the years 1982 to 2018. In our work, we propose extracting features by considering the atmosphere and land moisture and energy budgets and forecasting global droughts on a seasonal and an annual scale at 1, 3, 6, 9, 12 and 24 months lead times. SPI helps us to identify the severity and the duration of the drought to classify them as meteorological, agricultural and hydrological.

Authors: Akanksha Ahuja (NOA); Xin Rong Chua (Centre for Climate Research Singapore)
NeurIPS 2022 ForestBench: Equitable Benchmarks for Monitoring, Reporting, and Verification of Nature-Based Solutions with Machine Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Restoring ecosystems and reducing deforestation are necessary tools to mitigate the anthropogenic climate crisis. Current measurements of forest carbon stock can be inaccurate, in particular for underrepresented and small-scale forests in the Global South, hindering transparency and accountability in the Monitoring, Reporting, and Verification (MRV) of these ecosystems. There is thus need for high quality datasets to properly validate ML-based solutions. To this end, we present ForestBench, which aims to collect and curate geographically-balanced gold-standard datasets of small-scale forest plots in the Global South, by collecting ground-level measurements and visual drone imagery of individual trees. These equitable validation datasets for ML-based MRV of nature-based solutions shall enable assessing the progress of ML models for estimating above-ground biomass, ground cover, and tree species diversity.

Authors: Lucas Czech (Carnegie Institution for Science); Björn Lütjens (MIT); David Dao (ETH Zurich)
NeurIPS 2022 Estimating Heating Loads in Alaska using Remote Sensing and Machine Learning Methods (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Alaska and the larger Arctic region are in much greater need of decarbonization than the rest of the globe as a result of the accelerated consequences of climate change over the past ten years. Heating for homes and businesses accounts for over 75% of the energy used in the Arctic region. However, the lack of thorough and precise heating load estimations in these regions poses a significant obstacle to the transition to renewable energy. In order to accurately measure the massive heating demands in Alaska, this research pioneers a geospatial-first methodology that integrates remote sensing and machine learning techniques. Building characteristics such as height, size, year of construction, thawing degree days, and freezing degree days are extracted using open-source geospatial information in Google Earth Engine (GEE). These variables coupled with heating load forecasts from the AK Warm simulation program are used to train models that forecast heating loads on Alaska’s Railbelt utility grid. Our research greatly advances geospatial capability in this area and considerably informs the decarbonization activities currently in progress in Alaska.

Authors: Madelyn Gaumer (University of Washington); Nick Bolten (Paul G. Allen School of Computer Science and Engineering, University of Washington); Vidisha Chowdhury (Heinz College of Information Systems and Public Policy, Carnegie Mellon University); Philippe Schicker (Heinz College of Information Systems and Public Policy, Carnegie Mellon University); Shamsi Soltani (Department of Epidemiology and Population Health, Stanford University School of Medicine); Erin D Trochim (University of Alaska Fairbanks)
NeurIPS 2022 Interpretable Spatiotemporal Forecasting of Arctic Sea Ice Concentration at Seasonal Lead Times (Proposals Track)
Abstract and authors: (click to expand)

Abstract: There are many benefits from the accurate forecasting of Arctic sea ice, however existing models struggle to reliably predict sea ice concentration at long lead times. Many numerical models exist but can be sensitive to initial conditions, and while recent deep learning-based methods improve overall robustness, they either do not utilize temporal trends or rely on architectures that are not performant at learning long-term sequential dependencies. We propose a method of forecasting sea ice concentration using neural circuit policies, a form of continuous time recurrent neural architecture, which improve the learning of long-term sequential dependencies compared to existing techniques and offer the added benefits of adaptability to irregular sequence intervals and high interpretability.

Authors: Matthew Beveridge (Independent Researcher); Lucas Pereira (ITI, LARSyS, Técnico Lisboa)
NeurIPS 2022 Personalizing Sustainable Agriculture with Causal Machine Learning (Proposals Track) Best Paper: Proposals
Abstract and authors: (click to expand)

Abstract: To fight climate change and accommodate the increasing population, global crop production has to be strengthened. To achieve the "sustainable intensification" of agriculture, transforming it from carbon emitter to carbon sink is a priority, and understanding the environmental impact of agricultural management practices is a fundamental prerequisite to that. At the same time, the global agricultural landscape is deeply heterogeneous, with differences in climate, soil, and land use inducing variations in how agricultural systems respond to farmer actions. The "personalization" of sustainable agriculture with the provision of locally adapted management advice is thus a necessary condition for the efficient uplift of green metrics, and an integral development in imminent policies. Here, we formulate personalized sustainable agriculture as a Conditional Average Treatment Effect estimation task and use Causal Machine Learning for tackling it. Leveraging climate data, land use information and employing Double Machine Learning, we estimate the heterogeneous effect of sustainable practices on the field-level Soil Organic Carbon content in Lithuania. We thus provide a data-driven perspective for targeting sustainable practices and effectively expanding the global carbon sink.

Authors: Georgios Giannarakis (National Observatory of Athens); Vasileios Sitokonstantinou (National Observatory of Athens); Roxanne Suzette Lorilla (National Observatory of Athens); Charalampos Kontoes (National Observatory of Athens)
NeurIPS 2022 Disaster Risk Monitoring Using Satellite Imagery (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Natural disasters such as flood, wildfire, drought, and severe storms wreak havoc throughout the world, causing billions of dollars in damages, and uprooting communities, ecosystems, and economies. Unfortunately, flooding events are on the rise due to climate change and sea level rise. The ability to detect and quantify them can help us minimize their adverse impacts on the economy and human lives. Using satellites to study flood is advantageous since physical access to flooded areas is limited and deploying instruments in potential flood zones can be dangerous. We are proposing a hands-on tutorial to highlight the use of satellite imagery and computer vision to study natural disasters. Specifically, we aim to demonstrate the development and deployment of a flood detection model using Sentinel-1 satellite data. The tutorial will cover relevant fundamental concepts as well as the full development workflow of a deep learning-based application. We will include important considerations such as common pitfalls, data scarcity, augmentation, transfer learning, fine-tuning, and details of each step in the workflow. Importantly, the tutorial will also include a case study on how the application was used by authorities in response to a flood event. We believe this tutorial will enable machine learning practitioners of all levels to develop new technologies that tackle the risks posed by climate change. We expect to deliver the below learning outcomes: • Develop various deep learning-based computer vision solutions using hardware-accelerated open-source tools that are optimized for real-time deployment • Create an optimized pipeline for the machine learning development workflow • Understand different performance metrics for model evaluation that are relevant for real world datasets and data imbalances • Understand the public sector’s efforts to support climate action initiatives and point out where the audience can contribute

Authors: Kevin Lee (NVIDIA); Siddha Ganju (NVIDIA); Edoardo Nemni (UNOSAT)
NeurIPS 2022 Machine Learning for Predicting Climate Extremes (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Climate change has led to a rapid increase in the occurrence of extreme weather events globally, including floods, droughts, and wildfires. In the longer term, some regions will experience aridification while others will risk sinking due to rising sea levels. Typically, such predictions are done via weather and climate models that simulate the physical interactions between the atmospheric, oceanic, and land surface processes that operate at different scales. Due to the inherent complexity, these climate models can be inaccurate or computationally expensive to run, especially for detecting climate extremes at high spatiotemporal resolutions. In this tutorial, we aim to introduce the participants to machine learning approaches for addressing two fundamental challenges. We will walk the participants through a hands-on tutorial for predicting climate extremes relating to temperature and precipitation in 2 setups: (1) temporal forecasting: the goal is to predict climate variables into the future (both direct single step approaches and iterative approaches that roll out the model for several timesteps), and (2) spatial downscaling: the goal is to learn a mapping that transforms low-resolution outputs of climate models into high-resolution regional forecasts. Through introductory presentations and colab notebooks, we aim to expose the participants to (a) APIs for accessing and navigating popular repositories that host global climate data, such as the Copernicus data store, (b) identifying relevant datasets, including auxiliary data (e.g., other climate variables such as geopotential), (c) scripts for downloading and preprocessing relevant datasets, (d) algorithms for training machine learning models, (d) metrics for evaluating model performance, and (e) visualization tools for both the dataset and predicted outputs. The coding notebooks will be in Python. No prior knowledge of climate science is required.

Authors: Hritik Bansal (UCLA); Shashank Goel (University of California Los Angeles); Tung Nguyen (University of California, Los Angeles); Aditya Grover (UCLA)
AAAI FSS 2022 Employing Deep Learning to Quantify Power Plant Greenhouse Gas Emissions via Remote Sensing Data
Abstract and authors: (click to expand)

Abstract: Greenhouse gasses (GHG) emitted from fossil-fuel-burning power plants pose a global threat to climate and public health. GHG emissions degrade air quality and increase the frequency of natural disasters five-fold, causing 8.7 million deaths per year. Quantifying GHG emissions is crucial for the success of the planet. However, current methods to track emissions cost upwards of $520,000/plant. These methods are cost prohibitive for developing countries, and are not globally standardized, leading to inaccurate emissions reports from nations and companies. I developed a low-cost solution via an end-to-end deep learning pipeline that utilizes observations of emitted smoke plumes in satellite imagery to provide an accurate, precise system for quantifying power plant GHG emissions by segmentation of power plant smoke plumes, classification of the plant fossil fuels, and algorithmic prediction of power generation and CO2 emissions. The pipeline was able to achieve a segmentation Intersection Over Union (IoU) score of 0.841, fuel classification accuracy of 92%, and quantify power generation and CO2 emission rates with R2 values of 0.852 and 0.824 respectively. The results of this work serve as a step toward the low-cost monitoring and detection of major sources of GHG emissions, helping limit their catastrophic impacts on climate and our planet.

Authors: Aryan Jain (Amador Valley High School)
NeurIPS 2021 Flood Segmentation on Sentinel-1 SAR Imagery with Semi-Supervised Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Floods wreak havoc throughout the world, causing billions of dollars in damages, and uprooting communities, ecosystems and economies. The NASA Impact Emerging Techniques in Computational Intelligence (ETCI) competition on Flood Detection tasked participants with predicting flooded pixels after training with synthetic aperture radar (SAR) images in a supervised setting. We propose a semi-supervised learning pseudo-labeling scheme that derives confidence estimates from U-Net ensembles, thereby progressively improving accuracy. Concretely, we use a cyclical approach involving multiple stages (1) training an ensemble model of multiple U-Net architectures with the provided high confidence hand-labeled data and, generated pseudo labels or low confidence labels on the entire unlabeled test dataset, and then, (2) filter out quality generated labels and, (3) combine the generated labels with the previously available high confidence hand-labeled dataset. This assimilated dataset is used for the next round of training ensemble models. This cyclical process is repeated until the performance improvement plateaus. Additionally, we post process our results with Conditional Random Fields. Our approach sets a high score, and a new state-of-the-art on the Sentinel-1 dataset for the ETCI competition with 0.7654 IoU, an impressive improvement over the 0.60 IOU baseline. Our method, which we release with all the code including trained models, can also be used as an open science benchmark for the Sentinel-1 released dataset.

Authors: Siddha Ganju (Nvidia Corporation); Sayak Paul (Carted)
NeurIPS 2021 Towards Representation Learning for Atmospheric Dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: The prediction of future climate scenarios under anthropogenic forcing is critical to understand climate change and to assess the impact of potentially counter-acting technologies. Machine learning and hybrid techniques for this prediction rely on informative metrics that are sensitive to pertinent but often subtle influences. For atmospheric dynamics, a critical part of the climate system, no well established metric exists and visual inspection is currently still often used in practice. However, this ``eyeball metric'' cannot be used for machine learning where an algorithmic description is required. Motivated by the success of intermediate neural network activations as basis for learned metrics, e.g. in computer vision, we present a novel, self-supervised representation learning approach specifically designed for atmospheric dynamics. Our approach, called AtmoDist, trains a neural network on a simple, auxiliary task: predicting the temporal distance between elements of a randomly shuffled sequence of atmospheric fields (e.g. the components of the wind field from reanalysis or simulation). The task forces the network to learn important intrinsic aspects of the data as activations in its layers and from these hence a discriminative metric can be obtained. We demonstrate this by using AtmoDist to define a metric for GAN-based super resolution of vorticity and divergence. Our upscaled data matches both visually and in terms of its statistics a high resolution reference closely and it significantly outperform the state-of-the-art based on mean squared error. Since AtmoDist is unsupervised, only requires a temporal sequence of fields, and uses a simple auxiliary task, it has the potential to be of utility in a wide range of applications.

Authors: Sebastian Hoffmann (University of Magdeburg); Christian Lessig (Otto-von-Guericke-Universitat Magdeburg)
NeurIPS 2021 Memory to Map: Improving Radar Flood Maps With Temporal Context and Semantic Segmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global flood risk has increased due to worsening extreme weather events and human migration into growing flood-prone areas. Accurate, high-resolution, and near-real time flood maps can address flood risk by reducing financial loss and damage. We propose Model to Map, a novel machine learning approach that utilizes bi-temporal context to improve flood water segmentation performance for Sentinel-1 imagery. We show that the inclusion of unflooded context for the area, or "memory," allows the model to tap into a "prior state" of pre-flood conditions, increasing performance in geographic regions in which single-image radar-based flood mapping methods typically underperform (e.g. deserts). We focus on accuracy across different biomes to ensure global performance. Our experiments and novel data processing technique show that the confluence of pre-flood and permanent water context provides a 21% increase in mIoU over the baseline overall, and over 87% increase in deserts.

Authors: Veda Sunkara (Cloud to Street); Nicholas Leach (Cloud to Street); Siddha Ganju (Nvidia)
NeurIPS 2021 Global ocean wind speed estimation with CyGNSSnet (Papers Track)
Abstract and authors: (click to expand)

Abstract: The CyGNSS (Cyclone Global Navigation Satellite System) satellite system measures GNSS signals reflected off the Earth's surface. A global ocean wind speed dataset is derived, which fills a gap in Earth observation data, will improve cyclone forecasting, and could be used to mitigate effects of climate change. We propose CyGNSSnet, a deep learning model for predicting wind speed from CyGNSS observables, and evaluate its potential for operational use. With CyGNSSnet, performance improves by 29\% over the current operational model. We further introduce a hierarchical model, that combines an extreme value classifier and a specialized CyGNSSnet and slightly improves predictions for high winds.

Authors: Caroline Arnold (German Climate Computing Center); Milad Asgarimehr (German Research Centre for Geosciences)
NeurIPS 2021 Predicting Critical Biogeochemistry of the Southern Ocean for Climate Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: The Biogeochemical-Argo (BGC-Argo) program is building a network of globally distributed, sensor-equipped robotic profiling floats, improving our understanding of the climate system and how it is changing. These floats, however, are limited in the number of variables measured. In this study, we train neural networks to predict silicate and phosphate values in the Southern Ocean from temperature, pressure, salinity, oxygen, nitrate, and location and apply these models to earth system model (ESM) and BGC-Argo data to expand the utility of this ocean observation network. We trained our neural networks on observations from the Global Ocean Ship-Based Hydrographic Investigations Program (GO-SHIP) and use dropout regularization to provide uncertainty bounds around our predicted values. Our neural network significantly improves upon linear regression but shows variable levels of uncertainty across the ranges of predicted variables. We explore the generalization of our estimators to test data outside our training distribution from both ESM and BGC-Argo data. Our use of out-of-distribution test data to examine shifts in biogeochemical parameters and calculate uncertainty bounds around estimates advance the state-of-the-art in oceanographic data and climate monitoring. We make our data and code publicly available.

Authors: Ellen Park (MIT); Jae Deok Kim (MIT-WHOI); Nadege Aoki (MIT); Yumeng Cao (MIT); Yamin Arefeen (Massachusetts Institute of Technology); Matthew Beveridge (Massachusetts Institute of Technology); David P Nicholson (Woods Hole Oceanographic Institution); Iddo Drori (MIT)
NeurIPS 2021 WiSoSuper: Benchmarking Super-Resolution Methods on Wind and Solar Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: The transition to green energy grids depends on detailed wind and solar forecasts to optimize the siting and scheduling of renewable energy generation. Operational forecasts from numerical weather prediction models, however, only have a spatial resolution of 10 to 20-km, which leads to sub-optimal usage and development of renewable energy farms. Weather scientists have been developing super-resolution methods to increase the resolution, but often rely on simple interpolation techniques or computationally expensive differential equation-based models. Recently, machine learning-based models, specifically the physics-informed resolution-enhancing generative adversarial network (PhIREGAN), have outperformed traditional downscaling methods. We provide a thorough and extensible benchmark of leading deep learning-based super-resolution techniques, including the enhanced super-resolution generative adversarial network (ESRGAN) and an enhanced deep super-resolution (EDSR) network, on wind and solar data. We accompany the benchmark with a novel public, processed, and machine learning-ready dataset for benchmarking super-resolution methods on wind and solar data.

Authors: Rupa Kurinchi-Vendhan (Caltech); Björn Lütjens (MIT); Ritwik Gupta (University of California, Berkeley); Lucien D Werner (California Institute of Technology); Dava Newman (MIT); Steven Low (California Institute of Technology)
NeurIPS 2021 MS-nowcasting: Operational Precipitation Nowcasting with Convolutional LSTMs at Microsoft Weather (Papers Track)
Abstract and authors: (click to expand)

Abstract: We present the encoder-forecaster convolutional long short-term memory (LSTM) deep-learning model that powers Microsoft Weather's operational precipitation nowcasting product. This model takes as input a sequence of weather radar mosaics and deterministically predicts future radar reflectivity at lead times up to 6 hours. By stacking a large input receptive field along the feature dimension and conditioning the model's forecaster with predictions from the physics-based High Resolution Rapid Refresh (HRRR) model, we are able to outperform optical flow and HRRR baselines by 20-25% on multiple metrics averaged over all lead times.

Authors: Sylwester Klocek (Microsoft Corporation); Haiyu Dong (Microsoft); Matthew Dixon (Microsoft Corporation); Panashe Kanengoni (Microsoft Corporation); Najeeb Kazmi (Microsoft); Pete Luferenko (Microsoft Corporation); Zhongjian Lv (Microsoft Corporation); Shikhar Sharma (); Jonathan Weyn (Microsoft); Siqi Xiang (Microsoft Corporation)
NeurIPS 2021 Synthetic Imagery Aided Geographic Domain Adaptation for Rare Energy Infrastructure Detection in Remotely Sensed Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Object detection in remotely sensed data is frequently stymied by applications in geographies that are different from that of the training data. When objects are rare, the problem is exacerbated further. This is true of assessments of energy infrastructure such as generation, transmission, and end-use consumption; key to electrification planning as well as for effective assessment of natural disaster impacts which are varying in frequency and intensity due to climate change. We propose an approach to domain adaptation that requires only unlabeled samples from the target domain and generates synthetic data to augment training data for targeted domain adaptation. This approach is shown to work consistently across four geographically diverse domains, improving object detection average precision by 15.5\% on average for small sample sizes.

Authors: Wei Hu (Duke University); Tyler Feldman (Duke University); Eddy Lin (Duke University); Jose Luis Moscoso (Duke); Yanchen J Ou (Duke University); Natalie Tarn (Duke University); Baoyan Ye (Duke University); Wendy Zhang (Duke University); Jordan Malof (Duke University); Kyle Bradbury (Duke University)
NeurIPS 2021 Evaluating Pretraining Methods for Deep Learning on Geophysical Imaging Datasets (Papers Track)
Abstract and authors: (click to expand)

Abstract: Machine learning has the potential to automate the analysis of vast amounts of raw geophysical data, allowing scientists to monitor changes in key aspects of our climate such as cloud cover in real-time and at fine spatiotemporal scales. However, the lack of large labeled training datasets poses a significant barrier for effectively applying machine learning to these applications. Transfer learning, which involves first pretraining a neural network on an auxiliary “source” dataset and then finetuning on the “target” dataset, has been shown to improve accuracy for machine learning models trained on small datasets. Across prior work on machine learning for geophysical imaging, different choices are made about what data to pretrain on, and the impact of these choices on model performance is unclear. To address this, we systematically explore various settings of transfer learning for cloud classification, cloud segmentation, and aurora classification. We pretrain on different source datasets, including the large ImageNet dataset as well as smaller geophysical datasets that are more similar to the target datasets. We also experiment with multiple transfer learning steps where we pretrain on more than one source dataset. Despite the smaller source datasets’ similarity to the target datasets, we find that pretraining on the large, general-purpose ImageNet dataset yields significantly better results across all of our experiments. Transfer learning is especially effective for smaller target datasets, and in these cases, using multiple source datasets can give a marginal added benefit.

Authors: James Chen (Kirby School)
NeurIPS 2021 High-resolution rainfall-runoff modeling using graph neural network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Time-series modeling has shown great promise in recent studies using the latest deep learning algorithms such as LSTM (Long Short-Term Memory). These studies primarily focused on watershed-scale rainfall-runoff modeling or streamflow forecasting, but the majority of them only considered a single watershed as a unit. Although this simplification is very effective, it does not take into account spatial information, which could result in significant errors in large watersheds. Several studies investigated the use of GNN (Graph Neural Networks) for data integration by decomposing a large watershed into multiple sub-watersheds, but each sub-watershed is still treated as a whole, and the geoinformation contained within the watershed is not fully utilized. In this paper, we propose the GNRRM (Graph Neural Rainfall-Runoff Model), a novel deep learning model that makes full use of spatial information from high-resolution precipitation data, including flow direction and geographic information. When compared to baseline models, GNRRM has less over-fitting and significantly improves model performance. Our findings support the importance of hydrological data in deep learning-based rainfall-runoff modeling, and we encourage researchers to include more domain knowledge in their models.

Authors: Zhongrun Xiang (University of Iowa); Ibrahim Demir (The University of Iowa)
NeurIPS 2021 Machine Learning for Snow Stratigraphy Classification (Papers Track)
Abstract and authors: (click to expand)

Abstract: Snow-layer segmentation and classification is an essential diagnostic task for a wide variety of cryospheric science and climate research applications. To this end a Snow Micro Pen (SMP) can be used - a portable high-resolution snow penetrometer. However, the penetration-force measurements of the SMP must be labeled manually, which is a time-intensive task that requires training and becomes infeasible for large datasets. Here, we evaluate how well machine learning models can automatically segment and classify SMP profiles. Fourteen different models are trained on the MOSAiC SMP dataset, a unique and large SMP dataset of snow on Arctic sea-ice profiles. Depending on the user's task and needs, the long short-term memory neural network and the random forests are performing the best. The findings presented here facilitate and accelerate SMP data analysis and in consequence, help scientists to analyze the effects of climate change on the cryosphere more efficiently.

Authors: Julia Kaltenborn (McGill University); Viviane Clay (Osnabrück University); Amy R. Macfarlane (WSL Institute for Snow and Avalanche Research SLF); Martin Schneebeli (WSL Institute for Snow and Avalanche Research SLF)
NeurIPS 2021 DEM Super-Resolution with EfficientNetV2 (Papers Track)
Abstract and authors: (click to expand)

Abstract: Efficient climate change monitoring and modeling rely on high-quality geospatial and environmental datasets. Due to limitations in technical capabilities or resources, the acquisition of high-quality data for many environmental disciplines is costly. Digital Elevation Model (DEM) datasets are such examples whereas their low-resolution versions are widely available, high-resolution ones are scarce. In an effort to rectify this problem, we propose and assess an EfficientNetV2 based model. The proposed model increases the spatial resolution of DEMs up to 16 times without additional information.

Authors: Bekir Z Demiray (University of Iowa); Muhammed A Sit (The University of Iowa); Ibrahim Demir (The University of Iowa)
NeurIPS 2021 Towards Automatic Transformer-based Cloud Classification and Segmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Clouds have been demonstrated to have a huge impact on the energy balance, temperature, and weather of the Earth. Classification and segmentation of clouds and coverage factors is crucial for climate modelling, meteorological studies, solar energy industry, and satellite communication. For example, clouds have a tremendous impact on short-term predictions or 'nowcasts' of solar irradiance and can be used to optimize solar power plants and effectively exploit solar energy. However even today, cloud observation requires the intervention of highly-trained professionals to document their findings, which introduces bias. To overcome these issues and contribute to climate change technology, we propose, to the best of our knowledge, the first two transformer-based models applied to cloud data tasks. We use the CCSD Cloud classification dataset and achieve 90.06% accuracy, outperforming all other methods. To demonstrate the robustness of transformers in this domain, we perform Cloud segmentation on SWIMSWG dataset and achieve 83.2% IoU, also outperforming other methods. With this, we signal a potential shift away from pure CNN networks.

Authors: Roshan Roy (Birla Institute of Technology and Science, Pilani); Ahan M R (BITS Pilani); Vaibhav Soni (MANIT Bhopal); Ashish Chittora (BITS Pilani)
NeurIPS 2021 Machine learning-enabled model-data integration for predicting subsurface water storage (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Subsurface water storage (SWS) is a key variable of the climate system and a storage component for precipitation and radiation anomalies, inducing persistence in the climate system. It plays a critical role in climate-change projections and can mitigate the impacts of climate change on ecosystems. However, because of the difficult accessibility of the underground, hydrologic properties and dynamics of SWS are poorly known. Direct observations of SWS are limited, and accurate incorporation of SWS dynamics into Earth system land models remains challenging. We propose a machine learning-enabled model-data integration framework to improve the SWS prediction at local to conus scales in a changing climate by leveraging all the available observation and simulation resources, as well as to inform the model development and guide the observation collection. The accurate prediction will enable an optimal decision of water management and land use and improve the ecosystem's resilience to the climate change.

Authors: Dan Lu (Oak Ridge National Laboratory); Eric Pierce (Oak Ridge National Laboratory); Shih-Chieh Kao (Oak Ridge National Laboratory); David Womble (Oak Ridge National Laboratory); LI LI (Pennsylvania State University); Daniella Rempe (The University of Texas at Austin)
NeurIPS 2021 Unsupervised Machine Learning framework for sensor placement optimization: analyzing methane leaks (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane is one of the most potent greenhouse gases, with the global oil and gas industry being the second largest source of anthropogenic methane emissions, accounting for about 63% of the whole energy sector. This underscores the importance of detecting and remediating methane leaks for the entire oil and gas value chain. Methane sensor networks are a promising technology to detect methane leaks in a timely manner. While they provide near-real-time monitoring of an area of interest, the density of the network can be cost prohibitive, and the identification of the source of the leak is not apparent, especially where there could be more than one source. To address these issues, we developed a machine learning framework that leverages various data sources including oil and gas facilities data, historical methane leak rate distribution and meteorological data, to optimize sensor placement. The determination of sensor locations follows the objective to maximize the detection of possible methane leaks with a limited sensor budget.

Authors: Shirui Wang (University of Houston); Sara Malvar (Microsoft); Leonardo Nunes (Microsoft); Kim Whitehall (Microsoft); YAGNA DEEPIKA ORUGANTI (MICROSOFT); Yazeed Alaudah (Microsoft); Anirudh Badam (Microsoft)
NeurIPS 2021 Leveraging machine learning for identify hydrological extreme events under global climate change (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Hydrological extreme events, such as droughts and floods, are highly destructive natural disasters and its occurrence is expected to increase under the future climate change. Accurate and efficient approach to detect such events will provide timely information to assist management strategies for minimizing socio-economic damages. Despite the threshold approach has established to detect extreme events, the missing data from hydroclimate data and accurately identifying these events are still major challenges. The advent of machine learning models can help to identify the occurrence of droughts and floods events accurately and efficiently. Therefore, this proposed study will develop a machine learning model with semi-supervised anomaly detection approach to identify hydrological extreme events with ground-based data. As a test case, we will use 45-years record of hydroclimate data in coastal California, where was the driest region in 2012-2015, following with flash floods events. The expected results will increase communities’ awareness for hydrological extreme events and enable environmental planning and resource management under climate change

Authors: Ying-Jung C Deweese (Georgia Insititute of Technology)
NeurIPS 2021 DeepQuake: Artificial Intelligence for Earthquake Forecasting Using Fine-Grained Climate Data (Proposals Track) Best Paper: Proposals
Abstract and authors: (click to expand)

Abstract: Earthquakes are one of the most catastrophic natural disasters, making accurate, fine-grained, and real-time earthquake forecasting extremely important for the safety and security of human lives. In this work, we propose DeepQuake, a hybrid physics and deep learning model for fine-grained earthquake forecasting using time-series data of the horizontal displacement of earth’s surface measured from continuously operating Global Positioning System (cGPS) data. Recent studies using cGPS data have established a link between transient deformation within earth's crust to climate variables. DeepQuake’s physics-based pre-processing algorithm extracts relevant features including the x, y, and xy components of strain in earth’s crust, capturing earth’s elastic response to these climate variables, and feeds it into a deep learning neural network to predict key earthquake variables such as the time, location, magnitude, and depth of a future earthquake. Results across California show promising correlations between cGPS derived strain patterns and the earthquake catalog ground truth for a given location and time.

Authors: Yash Narayan (The Nueva School)