Time-series Analysis

Tutorials

Blog Posts

Workshop Papers

Venue Title
ICLR 2024 Improving Streamflow Predictions with Vision Transformers (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate streamflow prediction is crucial to understand climate impacts on water resources and develop effective adaption strategies. A global Long Short-Term Memory (LSTM) model, using data from multiple basins, can enhance streamflow prediction, yet acquiring detailed basin attributes remains a challenge. To overcome this, we introduce the Geo-ViT-LSTM model, a novel approach that enriches LSTM predictions by integrating basin attributes derived from remote sensing with a vision transformer. Applied to 531 basins across the United States (US), our method significantly outperforms existing models, showing an 11% increase in prediction accuracy. Geo-ViT-LSTM marks a significant advancement in land surface modeling, providing a more comprehensive and effective tool for managing water resources under climate change.

Authors: Kshitij Tayal (Oak Ridge National Lab); Arvind Renganathan (University of Minnesota); Dan Lu (Oak Ridge National Laboratory)
ICLR 2024 Extreme Precipitation Nowcasting using Transformer-based generative models (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper presents an innovative approach to extreme precipitation nowcasting by employing Transformer-based generative models, specifically VideoGPT with Extreme Value Loss (EVL) regularization. Leveraging a comprehensive dataset from the Royal Netherlands Meteorological Institute (KNMI), our study focuses on predicting short-term precipitation with high accuracy. We introduce a novel method for computing EVL without assuming fixed extreme representations, addressing the limitations of current models in capturing extreme weather events. We present both qualitative and quantitative analyses, demonstrating the superior performance of the proposed VideoGPT-EVL in generating accurate precipitation forecasts, especially when dealing with extreme precipitation events.

Authors: Cristian Meo (TUDelft); Mircea T Lica (Delft University of Technology); Ankush Roy (TUDelft); Zeina Boucher (TUDelft); Junzhe Yin (TUDelft); Yanbo Wang (Delft University of Technology); Ruben Imhoff (Deltares); Remko Uijlenhoet (TUDelft); Justin Dauwels (TU Delft)
ICLR 2024 GeoFormer: A Vision and Sequence Transformer-based Approach for Greenhouse Gas Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Air pollution represents a pivotal environmental challenge globally, playing a major role in climate change via greenhouse gas emissions and negatively affecting the health of billions. However, predicting the spatial and temporal patterns of pollutants remains challenging. The scarcity of ground-based monitoring facilities and the dependency of air pollution modeling on comprehensive datasets, often inaccessible for numerous areas, complicate this issue. In this work, we introduce GeoFormer, a compact model that combines a vision transformer module with a highly efficient time-series transformer module to predict surface-level nitrogen dioxide (NO2) concentrations from Sentinel-5P satellite imagery. We train the proposed model to predict surface-level NO2 measurements using a dataset we constructed with Sentinel-5P images of ground-level monitoring stations, and their corresponding NO2concentration readings. The proposed model attains high accuracy (MAE 5.65), demonstrating the efficacy of combining vision and time-series transformer architectures to harness satellite-derived data for enhanced GHG emission insights, proving instrumental in advancing climate change monitoring and emission regulation efforts globally.

Authors: Madhav Khirwar (Independent)
ICLR 2024 Time-Varying Constraint-Aware Reinforcement Learning for Energy Storage Control (Papers Track)
Abstract and authors: (click to expand)

Abstract: Energy storage devices, such as batteries, thermal energy storages, and hydrogen systems, can help mitigate climate change by ensuring a more stable and sustainable power supply. To maximize the effectiveness of such energy storage, determining the appropriate charging and discharging amounts for each time period is crucial. Reinforcement learning is preferred over traditional optimization for the control of energy storage due to its ability to adapt to dynamic and complex environments. However, the continuous nature of charging and discharging levels in energy storage poses limitations for discrete reinforcement learning, and time-varying feasible charge-discharge range based on state of charge (SoC) variability also limits the conventional continuous reinforcement learning. In this paper, we propose a continuous reinforcement learning approach that takes into account the time-varying feasible charge-discharge range. An additional objective function was introduced for learning the feasible action range for each time period, supplementing the objectives of training the actor for policy learning and the critic for value learning. This actively promotes the utilization of energy storage by preventing them from getting stuck in suboptimal states, such as continuous full charging or discharging. This is achieved through the enforcement of the charging and discharging levels into the feasible action range. The experimental results demonstrated that the proposed method further maximized the effectiveness of energy storage by actively enhancing its utilization.

Authors: Jaeik Jeong (Electronics and Telecommunications Research Institute (ETRI)); Tai-Yeon Ku (Electronics and Telecommunications Research Institute (ETRI)); Wan-Ki Park (Electronics and Telecommunications Research Institute (ETRI))
ICLR 2024 Reconstructing the Breathless Ocean with Spatio-Temporal Graph Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: The ocean is currently undergoing severe deoxygenation. Accurately reconstructing the breathless ocean is crucial for assessing and protecting marine ecosystem in response to climate change. Existing expert-dominated numerical simulations fail to catch up with the dynamic variation caused by global warming and human activities. Besides, due to the high-cost data collection, the historical observations are severely sparse, leading to big challenge for precise reconstruction. In this work, we propose OxyGenerator, the first spatio-temporal graph learning model, to reconstruct the global ocean deoxygenation from 1920 to 2023. Specifically, to address the heterogeneity across large temporal and spatial scales, we propose zoning-varying graph message-passing to capture the complex oceanographic correlations between missing values and sparse observations. Additionally, to further calibrate the uncertainty, we incorporate inductive bias from dissolved oxygen (DO) variations and chemical effects. Compared with in-situ DO observations, OxyGenerator significantly outperforms CMIP6 numerical simulations, reducing MAPE by 38.77%, demonstrating a promising potential to understand the ocean deoxygenation in data-driven manner.

Authors: Bin Lu (Shanghai Jiao Tong University); Ze Zhao (Shanghai Jiao Tong University); Luyu Han (Shanghai Jiao Tong University); Xiaoying Gan (Shanghai Jiao Tong University); Yuntao Zhou (Shanghai Jiao Tong University); Lei Zhou (Shanghai Jiao Tong Univ); Luoyi Fu (Shanghai Jiao Tong University); Xinbing Wang (Shanghai Jiao Tong University); Chenghu Zhou (Institute of Geographic Sciences and Natural Resources Research, CAS); Jing Zhang (Shanghai Jiao Tong University)
ICLR 2024 Neural Processes for Short-Term Forecasting of Weather Attributes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Traditional weather prediction models rely on solving complex physical equations, with long computation time. Machine learning models can process large amount of data more quickly. We propose to use neural processes (NPs) for short-term weather attributes forecasting. This is a novel avenue of research, as previous work has focused on NPs for long-term forecasting. We compare a multi-task neural process (MTNP) to an ensemble of independent single-task NPs (STNP) and to an ensemble of Gaussian processes (GPs). We use time series data for multiple weather attributes from Chichester Harbour over a one-week period. We evaluate performance in terms of NLL and MSE with 2-hours and 6-hours time horizons. When limited context information is provided, the MTNP leverages inter-task knowledge and outperforms the STNP. The STNP outperforms both the MTNP and the GPs ensemble when a sufficient, but not exceeding, amount of context information is provided.

Authors: Benedetta L Mussati (University of Oxford); Helen McKay (Mind Foundry); Stephen Roberts (University of Oxford)
ICLR 2024 EU Climate Change News Index: Forecasting EU ETS prices with online news (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon emission allowance prices have been rapidly increasing in the EU since 2018 and accurate forecasting of EU Emissions Trading System (ETS) prices has become essential. This paper proposes a novel method to generate alternative predictors for daily ETS price returns using relevant online news information. We devise the EU Climate Change News Index by calculating the term frequency–inverse document frequency (TF–IDF) feature for climate change-related keywords. The index is capable of tracking the ongoing debate about climate change in the EU. Finally, we show that incorporating the index in a simple predictive model significantly improves forecasts of ETS price returns.

Authors: Aron Pap (BGSE); Aron D Hartvig (Corvinus University of Budapest, Cambridge Econometrics); Péter Pálos (Budapest University of Technology and Economics)
ICLR 2024 On the potential of Optimal Transport in Geospatial Data Science (Papers Track)
Abstract and authors: (click to expand)

Abstract: Prediction problems in geographic information science and transportation are often motivated by the possibility to enhance operational efficiency and thereby reduce emissions. Examples range from predicting car sharing demand for relocation planning to forecasting traffic congestion for navigation purposes. However, conventional accuracy metrics ignore the spatial distribution of the errors, despite its relevance for operations. Here, we put forward a spatially aware evaluation metric and loss function based on Optimal Transport (OT). Our framework leverages partial OT and can minimize relocation costs in any spatial prediction problem. We showcase the advantages of OT-based evaluation over conventional metrics and further demonstrate the application of an OT loss function for improving forecasts of bike sharing demand and charging station occupancy. Thus, our framework not only aligns with operational considerations, but also signifies a step forward in refining predictions within geospatial applications. All code is available at https://github.com/mie-lab/geospatial_optimal_transport.

Authors: Nina V Wiedemann (ETH Zurich); Martin Raubal (ETH Zürich)
ICLR 2024 Generalizable Temperature Nowcasting with Physics-Constrained RNNs for Predictive Maintenance of Wind Turbine Components (Papers Track)
Abstract and authors: (click to expand)

Abstract: Machine learning plays an important role in the operation of current wind energy production systems. One central application is predictive maintenance to increase efficiency and lower electricity costs by reducing downtimes. Integrating physics-based knowledge in neural networks to enforce their physical plausibilty is a promising method to improve current approaches, but incomplete system information often impedes their application in real world scenarios. We describe a simple and efficient way for physics-constrained deep learning-based predictive maintenance for wind turbine gearbox bearings with partial system knowledge. The approach is based on temperature nowcasting constrained by physics, where unknown system coefficients are treated as learnable neural network parameters. Results show improved generalization performance to unseen environments compared to a baseline neural network, which is especially important in low data scenarios often encountered in real-world applications.

Authors: Johannes Exenberger (Graz University of Technology); Matteo Di Salvo (Sirius Energy Automation); Thomas Hirsch (Graz University of Technology); Franz Wotawa (Graz University of Technology, Institute for Software Technology); Gerald Schweiger (Graz University of Technology)
ICLR 2024 A cautionary tale about deep learning-based climate emulators (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models are computationally too expensive for many tasks, such as, rapidly exploring future impacts of climate policies. Thus, since the 1980s scientists have been developing lightweight approximations or emulators of climate models. Recently, deep learning has been proposed for this task and most commonly been evaluated on the benchmark ClimateBenchv1.0. We implemented a linear regression-based model from the 1990s with 30K parameters, called linear pattern scaling, that is now the 'best' model on ClimateBenchv1.0 -- outperforming the incumbent 100M-parameter foundation model, ClimaX, on the spatial error of 3 out of the 4 variables. Nevertheless, climate emulation might benefit from innovations in machine learning and we analyse two aspects that need to be addressed in future emulators: First, the data complexity depends strongly on the climate variable of interest and the chosen spatiotemporal resolution. Second, current benchmarks do not sufficiently address the large impact of interannual variability in the climate system. We have published our analysis as an interactive tutorial at github.com/ygaxolotl/tags-linear-pattern-scaling.

Authors: Björn Lütjens (Massachussets Institute of Technology); Raffaele Ferrari (Massachusetts Institute of Technology); Paolo Giani (Massachusetts Institute of Technology); Dava Newman (MIT); Andre Souza (Massachusetts Institute of Technology); Duncan Watson-Parris (University of California San Diego); Noelle Selin (Massachusetts Institute of Technology)
ICLR 2024 Identifying Complex Dynamics of Power Grid Frequency (Papers Track)
Abstract and authors: (click to expand)

Abstract: The energy system is undergoing rapid changes to integrate a growing number of intermittent renewable generators and facilitate the broader transition toward sustainability. As millions of consumers and thousands of (volatile) generators are connected to the same synchronous grid, no straightforward bottom-up models describing the dynamics are available on a continental scale comprising all of these necessary details. Hence, to identify this unknown power grid dynamics, we propose to leverage the Sparse Identification of Nonlinear Dynamics (SINDy) method. Thereby, we unveil the governing equations underlying the dynamical system directly from data measurements. Investigating the power grids of Iceland, Ireland and the Balearic islands as sample systems, we observe structurally similar dynamics with remarkable differences in both quantitative and qualitative behavior. Overall, we demonstrate how complex, i.e. non-linear, noisy, and time-dependent, dynamics can be identified straightforwardly.

Authors: Xinyi Wen (Karlsruhe Institute of Technology); Ulrich Oberhofer (Karlsruhe Institute of Technology); Leonardo Rydin Gorjão (Norwegian University of Life Sciences); G.Cigdem YALCIN (Istanbul University); Veit Hagenmeyer (Karlsruhe Institute of Technology (KIT)); Benjamin Schäfer (Karlsruhe Institute of Technology)
ICLR 2024 Valuation and Profit Allocation for Electric Vehicle Battery Data in a Data Market (Proposals Track)
Abstract and authors: (click to expand)

Abstract: This paper delves into the realm of electric vehicle (EV) battery data trading markets, focusing on data valuation and revenue allocation. In the face of fast-developing electric mobility, the safety of EV batteries becomes more and more important, driving the need for robust anomaly detection models. For newly found EV companies lacking extensive data, data markets offer a solution, facilitated by trading platforms. We shape this landscape, outline a transaction process involving data buyers, data sellers, and platforms. Our exploration extends to data valuation methodologies, encompassing the classic Shapley value and the least core algorithm. Considering the complicated mechanisms in EV battery, we unveil a deep learning framework for anomaly detection, treating EV batteries as dynamic systems. To explain data value from an economic perspective, we utilize a utility function considering the direct economic costs saved for the EV company to refine the evaluation process. Based on data value, we further propose revenue allocation schemes to allocate part of EV company's revenue to data sellers, offering diverse perspectives on fair and equitable profit distribution. A case study is conducted based on real world EV battery dataset to illustrate how the different revenue allocation schemes allocate payoffs to data sellers.

Authors: Junkang Chen (Peking University); Guannan He (Peking University)
ICLR 2024 An Adaptive Hydropower Management Approach for Downstream Ecosystem Preservation (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Hydropower plants play a pivotal role in advancing clean and sustainable energy production, contributing significantly to the global transition towards renewable energy sources. However, hydropower plants are currently perceived both positively as sources of renewable energy and negatively as disruptors of ecosystems. In this work, we highlight the overlooked potential of using hydropower plant as protectors of ecosystems by using adaptive ecological discharges. To advocate for this perspective, we propose using a neural network to predict the minimum ecological discharge value at each desired time. Additionally, we present a novel framework that seamlessly integrates it into hydropower management software, taking advantage of the well-established approach of using traditional constrained optimisation algorithms. This novel approach not only protects the ecosystems from climate change but also contributes to potentially increase the electricity production.

Authors: Cecília Coelho (University of Minho); Ming Jin (Virginia Tech); M. Fernanda P. Costa (Dep. Mathematics, University of Minho); Luís L. Ferrás (University of Porto)
ICLR 2024 Severe Wind Event Prediction with Multivariate Physics-Informed Deep Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Wind turbines play a crucial role in combating climate change by harnessing the force of the wind to generate clean and renewable energy. One key factor in ensuring the long-term effectiveness of wind turbines is the reduction of operating costs due to maintenance. Severe weather events, such as extreme changes in wind, can damage turbines, resulting in costly maintenance and economic losses in power production. We propose a preliminary physics-informed deep learning model to improve predictions of severe wind events and a multivariate time series extension for this work.

Authors: Willa Potosnak (Carnegie Mellon University); Cristian I Challu (Carnegie Mellon University); Kin G. Olivares (Carnegie Mellon University); James K Miller (Carnegie Mellon University); Artur Dubrawski (Carnegie Mellon University)
ICLR 2024 Planning for Floods & Droughts: Intro to AI-Driven Hydrological Modeling (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: This tutorial presents an AI-driven hydrological modeling approach to advance predictions of extreme hydrological events, including floods and droughts, which are of significant socioeconomic concerns. Traditionally, physics-based hydrological models have been the mainstay for simulating rainfall-runoff dynamics and forecasting streamflow. These models, while effective, are constrained by limitations in our systematic understanding and an inability to incorporate heterogeneous data. Recently, the surge in availability of multi-scale, multi-modal hydrological data has spurred the adoption of data-driven machine learning (ML) techniques. These methods have shown promising predictive performance. However, they often struggle with generalization and reliability, especially under climate change. This tutorial introduces physics-informed ML, by leveraging data and domain knowledge, to improve prediction accuracy and trustworthiness. We will delve into uncertainty quantification methods for probabilistic predictions that are vital for climate-resilient planning in managing floods and droughts. Participants will be guided through a comprehensive workflow, encompassing data analysis, model construction, and model evaluation. This tutorial is designed to elevate researchers’ understanding of hydrological systems and provide practitioners with robust, climate-resilient water management tools. These tools are instrumental in facilitating informed decision-making, crucial in the context of climate adaptation strategies. Participants will learn: ● Heterogeneous climate and hydrology data analysis ● State-of-the-art neural network models for rainfall-runoff modeling. ● ML model construction, training, validating, and testing ● Multiple ways to build a physics-informed ML model ● Uncertainty quantification in ML model predictions. All code and data will be publicly available for researchers/practitioners to build their own models.

Authors: Kshitij Tayal (Oak Ridge National Lab); Arvind Renganathan (University of Minnesota); Dan Lu (Oak Ridge National Laboratory)
ICLR 2024 Understanding drivers of climate extremes using regime-specific causal graphs (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: The climate system is intricate, involving numerous interactions among various components at multiple spatio-temporal scales. This complexity poses a significant challenge in understanding and predicting weather extremes within the Earth's climate system. However, a better understanding of the dynamics of such events is crucial due to their profound impact on ecosystems, economies, and worldwide communities. This tutorial will offer a comprehensive guide on using Regime-PCMCI (Saggioro et al., 2020), a constraint-based causal discovery technique, to uncover the causal relationships governing anomalous climate phenomena. Regime-PCMCI is designed to uncover causal relationships in time-series where transitions between regimes exist, and different causal relationships may govern each regime. In this tutorial, we will first discuss how to frame the problem of understanding climate and weather extremes using regime-specific causal discovery. We will shortly introduce constraint-based causal discovery and present the Regime-PCMCI algorithm. To enable participants to gain hands-on experience with the algorithm, we will apply Regime-PCMCI, implemented in the open-source Python package Tigramite (https://github.com/jakobrunge/tigramite), to a real-world climate science problem. Our example will focus on validating hypothesized regime-specific causal graphs that describe the causal relationship between atmospheric circulation, temperature, rainfall, evaporation, and soil moisture under various moisture regimes. Our tutorial will cover essential steps such as data preprocessing, parameter selection, and interpretation of results, ensuring that all participants with a basic understanding of climate science or data analysis can grasp the presented concepts. With this tutorial, we wish to equip participants with the skills to apply Regime-PCMCI in their research to further uncover complex mechanisms in climate science, as this knowledge is crucial for more informed policy-making.

Authors: Oana-Iuliana Popescu (Institute of Data Science, German Aerospace Center (DLR)); Wiebke Günther (German Aerospace Center); Raed Hamed (Institute for Environmental Studies, VU Amsterdam); Dominik Schumacher (4Institute for Atmospheric and Climate Science, ETH Zürich); Martin Rabel (DLR); Dim Coumou (IVM/VU); Jakob Runge (Institute of Data Science, German Aerospace Center (DLR))
NeurIPS 2023 Graph-based Neural Weather Prediction for Limited Area Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: The rise of accurate machine learning methods for weather forecasting is creating radical new possibilities for modeling the atmosphere. In the time of climate change, having access to high-resolution forecasts from models like these is also becoming increasingly vital. While most existing Neural Weather Prediction (NeurWP) methods focus on global forecasting, an important question is how these techniques can be applied to limited area modeling. In this work we adapt the graph-based NeurWP approach to the limited area setting and propose a multi-scale hierarchical model extension. Our approach is validated by experiments with a local model for the Nordic region.

Authors: Joel Oskarsson (Linköping University); Tomas Landelius (SMHI); Fredrik Lindsten (Linköping University)
NeurIPS 2023 Can Deep Learning help to forecast deforestation in the Amazonian Rainforest? (Papers Track)
Abstract and authors: (click to expand)

Abstract: Deforestation is a major driver of climate change. To mitigate deforestation, carbon offset projects aim to protect forest areas at risk. However, existing literature shows that most projects have substantially overestimated the risk of deforestation, thereby issuing carbon credits without equivalent emissions reductions. In this study, we examine if the spread of deforestation can be predicted ex-ante using Deep Learning (DL) models. Our input data includes past deforestation development, slope information, land use, and other terrain- and soil-specific covariates. Testing predictions 1-year ahead, we find that our models only achieve low levels of predictability. For pixel-wise classification at a 30 m resolution, our models achieve an F1 score of 0.263. Only when substantially simplifying the task to predicting if any level of deforestation occurs within a 1.5 km squared tile, the model results improve to a moderate performance (F1: 0.608). We conclude that, based on our input data, deforestation cannot be predicted accurately enough to justify the ex-ante issuance of carbon credits for forest conservation projects. As main challenges, there is the extreme class imbalance between pixels that are deforested (minority) and not deforested (majority) as well as the omittance of social, political, and economic drivers of deforestation.

Authors: Tim Engelmann (ETH Zurich); Malte Toetzke (ETH Zurich)
NeurIPS 2023 Machine learning derived sub-seasonal to seasonal extremes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Improving the accuracy of sub-seasonal to seasonal (S2S) extremes can significantly impact society. Providing S2S forecasts in risk or extreme indices can aid disaster response, especially for drought and flood events. Additionally, it can provide updates on disease outbreaks and aid in predicting the occurrence, duration, and decline of heat waves. This work uses a transformer model to predict the daily temperature distributions in the S2S scale. We analyze how the model performs in extreme temperatures by comparing its output distributions with those obtained from ECMWF forecasts across different metrics. Our model produces better responses for temperatures in average and extreme regions. Also, we show how our model better captures the heatwave that hit Europe in the summer of 2019.

Authors: Daniel Salles Civitarese (IBM Research, Brazil); Bianca Zadrozny (IBM Research)
NeurIPS 2023 An LSTM-based Downscaling Framework for Australian Precipitation Projections (Papers Track)
Abstract and authors: (click to expand)

Abstract: Understanding potential changes in future rainfall and their local impacts on Australian communities can inform adaptation decisions worth billions of dollars in insurance, agriculture, and other sectors. This understanding relies on downscaling a large ensemble of coarse Global Climate Models (GCMs), our primary tool for simulating future climate. However, the prohibitively high computational cost of downscaling has been a significant barrier. In response, this study develops a cost-efficient downscaling framework for daily precipitation using Long Short-Term Memory (LSTM) models. The models are trained with ERA5 reanalysis data and a customized quantile loss function to better capture precipitation extremes. The framework is employed to downscale precipitation from a GCM member of the CMIP6 ensemble. We demonstrate the skills of the downscaling models to capture spatial and temporal characteristics of precipitation. We also explore regional future changes in precipitation extremes projected by the downscaled GCM. In general, this framework will enable the generation of a large ensemble of regional future projections for Australian rainfall. This will further enhance the assessment of likely climate risks and the quantification of their uncertainties.

Authors: Matthias Bittner (Vienna University of Technology); Sanaa Hobeichi (The University of New South Wales); Muhammad Zawish (Walton Institute, WIT); Samo DIATTA (Assane Seck University of Ziguinchor); Remigius Ozioko (University of Nigeria); Sharon Xu (Indigo Ag); Axel Jantsch (TU Wien)
NeurIPS 2023 CityTFT: Temporal Fusion Transformer for Urban Building Energy Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Urban Building Energy Modeling (UBEM) is an emerging method to investigate urban design and energy systems against the increasing energy demand at urban and neighborhood levels. However, current UBEM methods are mostly physic-based and time-consuming in multiple climate change scenarios. This work proposes CityTFT, a data-driven UBEM framework, to accurately model the energy demands in urban environments. With the empowerment of the underlying TFT framework and an augmented loss function, CityTFT could predict heating and cooling triggers in unseen climate dynamics with an F1 score of 99.98 \% while RMSE of loads of 13571.3750 Wh.

Authors: Ting-Yu Dai (The University of Texas at Austin); Dev Niyogi (The University of Texas at Austin); Zoltan Nagy (The University of Texas at Austin)
NeurIPS 2023 Continuous Convolutional Neural Networks for Disruption Prediction in Nuclear Fusion Plasmas (Papers Track)
Abstract and authors: (click to expand)

Abstract: Grid decarbonization for climate change requires dispatchable carbon-free energy like nuclear fusion. The tokamak concept offers a promising path for fusion, but one of the foremost challenges in implementation is the occurrence of energetic plasma disruptions. In this study, we delve into Machine Learning approaches to predict plasma state outcomes. Our contributions are twofold: (1) We present a novel application of Continuous Convolutional Neural Networks for disruption prediction and (2) We examine the advantages and disadvantages of continuous models over discrete models for disruption prediction by comparing our model with the previous, discrete state of the art, and show that continuous models offer significantly better performance (Area Under the Receiver Operating Characteristic Curve = 0.974 v.s. 0.799) with fewer parameters.

Authors: William F Arnold (KAIST); Lucas Spangher (MIT PSFC); Cristina Rea (MIT PSFC)
NeurIPS 2023 Asset Bundling for Wind Power Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: The growing penetration of intermittent, renewable generation in US power grids results in increased operational uncertainty. In that context, accurate forecasts are critical, especially for wind generation, which exhibits large variability and is historically harder to predict. To overcome this challenge, this work proposes a novel Bundle-Predict-Reconcile (BPR) framework that integrates asset bundling, machine learning, and forecast reconciliation techniques to accurately predict wind power at the asset, bundle, and fleet level. Notably, our approach effectively introduces an auxiliary learning task (predicting the bundle-level time series) to help the main learning tasks (fleet-level time series) and proposes new asset-bundling criteria to capture the spatio-temporal dynamics of wind power time series. Extensive numerical experiments are conducted on an industry-size dataset of wind farms, demonstrating the benefits of BPR, which consistently and significantly improves forecast accuracy over the baseline approach, especially at the fleet level.

Authors: Hanyu Zhang (Georgia Institute of Technology); Mathieu Tanneau (Georgia Institute of Technology); Chaofan Huang (Georgia Institute of Technology); Roshan Joseph (Georgia Institute of Technology); Shangkun Wang (Georgia Institute of Technology); Pascal Van Hentenryck (Georgia Institute of Technology)
NeurIPS 2023 Monitoring Sustainable Global Development Along Shared Socioeconomic Pathways (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Sustainable global development is one of the most prevalent challenges facing the world today, hinging on the equilibrium between socioeconomic growth and environmental sustainability. We propose approaches to monitor and quantify sustainable development along the Shared Socioeconomic Pathways (SSPs), including mathematically derived scoring algorithms, and machine learning methods. These integrate socioeconomic and environmental datasets, to produce an interpretable metric for SSP alignment. An initial study demonstrates promising results, laying the groundwork for the application of different methods to the monitoring of sustainable global development.

Authors: Michelle Wan (University of Cambridge); Jeff Clark (University of Bristol); Edward Small (Royal Melbourne Institute of Technology); Elena Fillola (University of Bristol); Raul Santos Rodriguez (University of Bristol)
ICLR 2023 CityLearn: A Tutorial on Reinforcement Learning Control for Grid-Interactive Efficient Buildings and Communities (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Buildings are responsible for up to 75% of electricity consumption in the United States. Grid-Interactive Efficient Buildings can provide flexibility to solve the issue of power supply-demand mismatch, particularly brought about by renewables. Their high energy efficiency and self-generating capabilities can reduce demand without affecting the building function. Additionally, load shedding and shifting through smart control of storage systems can further flatten the load curve and reduce grid ramping cost in response to rapid decrease in renewable power supply. The model-free nature of reinforcement learning control makes it a promising approach for smart control in grid-interactive efficient buildings, as it can adapt to unique building needs and functions. However, a major challenge for the adoption of reinforcement learning in buildings is the ability to benchmark different control algorithms to accelerate their deployment on live systems. CityLearn is an open source OpenAI Gym environment for the implementation and benchmarking of simple and advanced control algorithms, e.g., rule-based control, model predictive control or deep reinforcement learning control thus, provides solutions to this challenge. This tutorial leverages CityLearn to demonstrate different control strategies in grid-interactive efficient buildings. Participants will learn how to design three controllers of varying complexity for battery management using a real-world residential neighborhood dataset to provide load shifting flexibility. The algorithms will be evaluated using six energy flexibility, environmental and economic key performance indicators, and their benefits and shortcomings will be identified. By the end of the tutorial, participants will acquire enough familiarity with the CityLearn environment for extended use in new datasets or personal projects.

Authors: Kingsley E Nweye (The University of Texas at Austin); Allen Wu (The University of Texas at Austin); Hyun Park (The University of Texas at Austin); Yara Almilaify (The University of Texas at Austin); Zoltan Nagy (The University of Texas at Austin)
ICLR 2023 Smart Meter Data Analytics: Practical Use-Cases and Best Practices of Machine Learning Applications for Energy Data in the Residential Sector (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: To cope with climate change, the energy system is undergoing a massive transformation. With the electrification of all sectors, the power grid is facing high additional demand. As a result, the digitization of the grid is becoming more of a focus. The smart grid relies heavily on the increasing deployment of smart electricity meters around the world. The corresponding smart meter data is typically a time series of power or energy measurements with a resolution of 1s to 60 min. This data provides valuable insights and opportunities for monitoring and controlling activities in the power grid. In this tutorial, we therefore provide an overview of best practices for analyzing smart meter data. We focus on machine learning applications and low resolution (15-60 minutes) energy data in a residential setting. We only use real-world datasets and cover use-cases that are highly relevant for practical applications. Although this tutorial is specifically tailored to an audience from the energy domain, we believe that anyone from the data analytics and machine learning community can benefit from it, as many techniques are applicable to any time series data. Through our tutorial, we hope to foster new ideas, contribute to an interdisciplinary exchange between different research fields, and educate people about energy use.

Authors: Tobias Brudermueller (ETH Zurich); Markus Kreft (ETH Zurich)
ICLR 2023 Long-lead forecasts of wintertime air stagnation index in southern China using oceanic memory effects (Papers Track)
Abstract and authors: (click to expand)

Abstract: Stagnant weather condition is one of the major contributors to air pollution as it is favorable for the formation and accumulation of pollutants. To measure the atmosphere’s ability to dilute air pollutants, Air Stagnation Index (ASI) has been introduced as an important meteorological index. Therefore, making long-lead ASI forecasts is vital to make plans in advance for air quality management. In this study, we found that autumn Niño indices derived from sea surface temperature (SST) anomalies show a negative correlation with wintertime ASI in southern China, offering prospects for a prewinter forecast. We developed an LSTM-based model to predict the future wintertime ASI. Results demonstrated that multivariate inputs (past ASI and Niño indices) achieve better forecast performance than univariate input (only past ASI). The model achieves a correlation coefficient of 0.778 between the actual and predicted ASI, exhibiting a high degree of consistency.

Authors: Chenhong Zhou (Hong Kong Baptist University); Xiaorui Zhang (Hong Kong Baptist University); Meng Gao (Hong Kong Baptist University); Shanshan Liu (University of science and technology of China); Yike Guo (Hong Kong University of Science and Technology); Jie Chen (Hong Kong Baptist University)
ICLR 2023 Attention-based Domain Adaptation Forecasting of Streamflow in Data-Sparse Regions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Streamflow forecasts are critical to guide water resource management, mitigate drought and flood effects, and develop climate-smart infrastructure and governance. Many global regions, however, have limited streamflow observations to guide evidence-based management strategies. In this paper, we propose an attention-based domain adaptation streamflow forecaster for data-sparse regions. Our approach leverages the hydrological characteristics of a data-rich source domain to induce effective 24hr lead-time streamflow prediction in a data-constrained target domain. Specifically, we employ a deep-learning framework leveraging domain adaptation techniques to simultaneously train streamflow predictions and discern between both domains using an adversarial method. Experiments against baseline cross-domain forecasting models show improved performance for 24hr lead-time streamflow forecasting.

Authors: Roland R Oruche (University of Missouri-Columbia); Fearghal O'Donncha (IBM Research)
ICLR 2023 BurnMD: A Fire Projection and Mitigation Modeling Dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: Today's fire projection modeling tools struggle to keep up with the rapid rate and increasing severity of climate change, leaving disaster managers dependent on tools which are increasingly unrepresentative of complex interactions between fire behavior, environmental conditions, and various mitigation options. This has consequences for equitably minimizing wildfire risks to life, property, ecology, cultural heritage, and public health. Fortunately, decades of data exist for fuel populations, weather conditions, and outcomes of significant fires in the American West and globally. The fire management community faces a lack of data standardization and validation among many competing fire models. Likewise, the machine learning community lacks curated datasets and benchmarks to develop solutions necessary to generate impact in this space. We present a novel dataset composed of 308 medium sized fires from the years 2018-2021, complete with both time series airborne based inference and ground operational estimation of fire extent, and operational mitigation data such as control line construction. As the first large wildfire dataset with mitigation information, Burn Mitigation Dataset (BurnMD) will help advance fire projection modeling, fire risk modeling, and AI generated land management policies.

Authors: Marissa Dotter (MITRE Corporation)
ICLR 2023 Improving a Shoreline Forecasting Model with Symbolic Regression (Papers Track)
Abstract and authors: (click to expand)

Abstract: Given the current context of climate change and the increasing population densities at coastal zones around the globe, there is an increasing need to be able to predict the development of our coasts. Recent advances in artificial intelligence allow for automatic analysis of observational data. Symbolic Regression (SR) is a type of Machine Learning algorithm that aims to find interpretable symbolic expressions that can explain relations in the data. In this work, we aim to study the problem of forecasting shoreline change using SR. We make use of Cartesian Genetic Programming (CGP) in order to encode and improve upon ShoreFor, a physical shoreline prediction model. During training, CGP individuals are evaluated and selected according to their predictive score at five different coastal sites. This work presents a comparison between a CGP-evolved model and the base ShoreFor model. In addition to evolution's ability to produce well-performing models, it demonstrates the usefulness of SR as a research tool to gain insight into the behaviors of shorelines in various geographical zones.

Authors: Mahmoud AL NAJAR (Laboratory of Spatial Geophysics and Oceanography Studies); Rafael ALMAR (Laboratory of Spatial Geophysics and Oceanography Studies); Erwin BERGSMA (CNES); Jean-Marc DELVIT (CNES); Dennis Wilson (ISAE)
ICLR 2023 Topology Estimation from Voltage Edge Sensing for Resource-Constrained Grids (Papers Track)
Abstract and authors: (click to expand)

Abstract: Electric grids are the conduit for renewable energy delivery and will play a crucial role in mitigating climate change. To do so successfully in resource-constrained low- and middle-income countries (LMICs), increasing operational efficiency is key. Such efficiency demands evolving knowledge of the grid’s state, of which topology---how points on the network are physically connected---is fundamental. In LMICs, knowledge of distribution topology is limited and established methods for topology estimation rely on expensive sensing infrastructure, such as smart meters or PMUs, that are inaccessible at scale. This paper lays the foundation for topology estimation from more accessible data: outlet-level voltage magnitude measurements. It presents a graph-based algorithm and explanatory visualization using the Fielder vector for estimating and communicating topological proximity from this data. We demonstrate the method on a real dataset collected in Accra, Ghana, thus opening the possibility of globally accessible, cutting-edge grid monitoring through non-traditional sensing strategies coupled with ML.

Authors: Mohini S Bariya (nLine); Genevieve Flaspohler (nLine)
ICLR 2023 Chasing Low-Carbon Electricity for Practical and Sustainable DNN Training (Papers Track)
Abstract and authors: (click to expand)

Abstract: Deep learning has experienced significant growth in recent years, resulting in increased energy consumption and carbon emission from the use of GPUs for training deep neural networks (DNNs). Answering the call for sustainability, conventional solutions have attempted to move training jobs to locations or time frames with lower carbon intensity. However, moving jobs to other locations may not always be feasible due to large dataset sizes or data regulations. Moreover, postponing training can negatively impact application service quality because the DNNs backing the service are not updated in a timely fashion. In this work, we present a practical solution that reduces the carbon footprint of DNN training without migrating or postponing jobs. Specifically, our solution observes real-time carbon intensity shifts during training and controls the energy consumption of GPUs, thereby reducing carbon footprint while maintaining training performance. Furthermore, in order to proactively adapt to shifting carbon intensity, we propose a lightweight machine learning algorithm that predicts the carbon intensity of the upcoming time frame. Our solution, Chase, reduces the total carbon footprint of training ResNet-50 on ImageNet by 13.6% while only increasing training time by 2.5%.

Authors: Zhenning Yang (University of Michigan, Ann Arbor); Luoxi Meng (University of Michigan, Ann Arbor); Jae-Won Chung (University of Michigan, Ann Arbor); Mosharaf Chowdhury (University of Michigan, Ann Arbor)
ICLR 2023 Graph-Based Deep Learning for Sea Surface Temperature Forecasts (Papers Track)
Abstract and authors: (click to expand)

Abstract: Sea surface temperature (SST) forecasts help with managing the marine ecosystem and the aquaculture impacted by anthropogenic climate change. Numerical dynamical models are resource intensive for SST forecasts; machine learning (ML) models could reduce high computational requirements and have been in the focus of the research community recently. ML models normally require a large amount of data for training. Environmental data are collected on regularly-spaced grids, so early work mainly used grid-based deep learning (DL) for prediction. However, both grid data and the corresponding DL approaches have inherent problems. As geometric DL has emerged, graphs as a more generalized data structure and graph neural networks (GNNs) have been introduced to the spatiotemporal domains. In this work, we preliminarily explored graph re-sampling and GNNs for global SST forecasts, and GNNs show better one month ahead SST prediction than the persistence model in most oceans in terms of root mean square errors.

Authors: Ding Ning (University of Canterbury); Varvara Vetrova (University of Canterbury); Karin Bryan (University of Waikato)
ICLR 2023 Uncovering the Spatial and Temporal Variability of Wind Resources in Europe: A Web-Based Data-Mining Tool (Papers Track)
Abstract and authors: (click to expand)

Abstract: We introduce REmap-eu.app, a web-based data-mining visualization tool of the spatial and temporal variability of wind resources. It uses the latest open-access dataset of the daily wind capacity factor in 28 European countries between 1979 and 2019 and proposes several user-configurable visualizations of the temporal and spatial variations of the wind power capacity factor. The platform allows for a deep analysis of the distribution, the cross-country correlation, and the drivers of low wind power events. It offers an easy-to-use interface that makes it suitable for the needs of researchers and stakeholders. The tool is expected to be useful in identifying areas of high wind potential and possible challenges that may impact the large-scale deployment of wind turbines in Europe. Particular importance is given to the visualization of low wind power events and to the potential of cross-border cooperations in mitigating the variability of wind in the context of increasing reliance on weather-sensitive renewable energy sources.

Authors: Alban Puech (École Polytechnique); Jesse Read (Ecole Polytechnique)
ICLR 2023 Robustly modeling the nonlinear impact of climate change on agriculture by combining econometrics and machine learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to have a dramatic impact on agricultural production; however, due to natural complexity, the exact avenues and relative strengths by which this will happen are still unknown. The development of accurate forecasting models is thus of great importance to enable policy makers to design effective interventions. To date, most machine learning methods aimed at tackling this problem lack a consideration of causal structure, thereby making them unreliable for the types of counterfactual analysis necessary when making policy decisions. Econometrics has developed robust techniques for estimating cause-effect relations in time-series, specifically through the use of cointegration analysis and Granger causality. However, these methods are frequently limited in flexibility, especially in the estimation of nonlinear relationships. In this work, we propose to integrate the non-linear function approximators with the robust causal estimation methods to ultimately develop an accurate agricultural forecasting model capable of robust counterfactual analysis. This method would be a valuable new asset for government and industrial stakeholders to understand how climate change impacts agricultural production.

Authors: Benedetta Francesconi (Independent Researcher); Ying-Jung C Deweese (Descartes Labs / Georgia Insititute of Technology)
ICLR 2023 EfficientTempNet: Temporal Super-Resolution of Radar Rainfall (Papers Track)
Abstract and authors: (click to expand)

Abstract: Rainfall data collected by various remote sensing instruments such as radars or satellites has different space-time resolutions. This study aims to improve the temporal resolution of radar rainfall products to help with more accurate climate change modeling and studies. In this direction, we introduce a solution based on EfficientNetV2, namely EfficientTempNet, to increase the temporal resolution of radar-based rainfall products from 10 minutes to 5 minutes. We tested EfficientRainNet over a dataset for the state of Iowa, US, and compared its performance to three different baselines to show that EfficientTempNet presents a viable option for better climate change monitoring.

Authors: Bekir Z Demiray (University of Iowa); Muhammed A Sit (The University of Iowa); Ibrahim Demir (University of Iowa)
ICLR 2023 Projecting the climate penalty on pm2.5 pollution with spatial deep learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The climate penalty measures the effects of a changing climate on air quality due to the interaction of pollution with climate factors, independently of future changes in emissions. This work introduces a statistical framework for estimating the climate penalty on soot pollution (PM 2.5), which has been linked to respiratory and cardiovascular diseases and premature mortality. The framework evaluates the disparities in future PM 2.5 exposure across racial/ethnic and income groups---an important step towards informing mitigation public health policy and promoting environmental equity in addressing the effects of climate change. The proposed methodology aims to improve existing statistical-based methods for estimating the climate penalty using an expressive and scalable predictive model based on spatial deep learning with spatiotemporal trend estimation. The proposed approach will (1) use higher-resolution climate inputs, which current statistical methods to estimate the climate penalty approaches cannot accommodate; (2) integrate additional predictive data sources such as demographics, geology, and land use; (3) consider regional dependencies and synoptic weather patterns influencing PM 2.5, deconvolving the effects of climate change from increasing air quality regulations and other sources of unmeasured spatial heterogeneity.

Authors: Mauricio Tec (Harvard University); Riccardo Cadei (Harvard University); Francesca Dominici (Harvard University); Corwin Zigler (University of Texas at Austin)
ICLR 2023 On the impact of small-data diversity on forecasts: evidence from meteorologically-driven electricity demand in Mediterranean zones. (Papers Track)
Abstract and authors: (click to expand)

Abstract: In this paper, we compare the improvement of probabilistic electricity demand forecasts for three specific coastal and island regions using raw and pre-computed meteorological features based on empirically-tested formulations drawn from climate science literature. Typically for the general task of time-series forecasting with strong weather/climate drivers, go-to models like the Autoregressive Integrated Moving Average (ARIMA) model are built with assumptions of how independent variables will affect a dependent one and are at best encoded with a handful of exogenous features with known impact. Depending on the geographical region and/or cultural practices of a population, such a selection process may yield a non-optimal feature set which would ultimately drive a weak impact on underline demand forecasts. The aim of this work is to assess the impact of a documented set of meteorological features on electricity demand using deep learning models in comparative studies. Leveraging the defining computational architecture of the Temporal Fusion Transformer (TFT), we discover the unimportance of weather features for improving probabilistic forecasts for the targeted regions. However, through experimentation, we discover that the more stable electricity demand of the coastal Mediterranean regions, the Ceuta and Melilla autonomous cities in Morocco, improved the forecast accuracy of the strongly tourist-driven electricity demand for the Balearic islands located in Spain during the time of travel restrictions (i.e., during COVID19 (2020))--a root mean squared error (RMSE) from ~0.090 to ~0.012 with a substantially improved 10th/90th quantile bounding.

Authors: Reginald Bryant (IBM Research - Africa); Julian Kuehnert (IBM Research)
NeurIPS 2022 Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Short-term forecasting of solar photovoltaic energy (PV) production is important for powerplant management. Ideally these forecasts are equipped with error bars, so that downstream decisions can account for uncertainty. To produce predictions with error bars in this setting, we consider Gaussian processes (GPs) for modelling and predicting solar photovoltaic energy production in the UK. A standard application of GP regression on the PV timeseries data is infeasible due to the large data size and non-Gaussianity of PV readings. However, this is made possible by leveraging recent advances in scalable GP inference, in particular, by using the state-space form of GPs, combined with modern variational inference techniques. The resulting model is not only scalable to large datasets but can also handle continuous data streams via Kalman filtering.

Authors: So Takao (UCL); Sean Nassimiha (UCL); Peter Dudfield (Open Climate Fix); Jack Kelly (Open Climate Fix); Marc Deisenroth (University College London)
NeurIPS 2022 Optimizing Japanese dam reservoir inflow forecast for efficient operation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Despite a climate and topology favorable to hydropower (HP) generation, HP only accounts for 4% of today’s Japanese primary energy consumption mix. In recent years, calls for improving the efficiency of Japanese HP towards achieving a more sustainable energy mix have emerged from prominent voices in the Ministry of Land, Infrastructure, Transport and Tourism (MILT). Among potential optimizations, data-driven dam operation policies using accurate river discharge forecasts have been advocated for. In the meantime, Machine Learning (ML) has recently made important strides in hydrological modeling, with forecast accuracy improvements demonstrated on both precipitation nowcasting and river discharge prediction. We are motivated by the convergence of these societal and technological contexts: our final goal is to provide scientific evidence and actionable insights for dam infrastructure managers and policy makers to implement more energy-efficient and flood-resistant dam operation policies on a national scale. Towards this goal this work presents a preliminary study of ML-based dam inflow forecasts on a dataset of 127 Japanese public dams we assembled. We discuss our preliminary results and lay out a path for future studies.

Authors: Keisuke Yoshimi (Kobe University); Tristan E.M Hascoet (Kobe University); Rousslan F. Julien Dossa (Kobe University); Ryoichi Takashima (Kobe University); Tetsuya Takiguchi (Kobe University); Satoru Oishi (Kobe University)
NeurIPS 2022 Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation (Papers Track)
Abstract and authors: (click to expand)

Abstract: How can we learn from all available data when training machine-learnt climate models, without incurring any extra cost at simulation time? Typically, the training data comprises coarse-grained high-resolution data. But only keeping this coarse-grained data means the rest of the high-resolution data is thrown out. We use a transfer learning approach, which can be applied to a range of machine learning models, to leverage all the high-resolution data. We use three chaotic systems to show it stabilises training, gives improved generalisation performance and results in better forecasting skill. Our code is at https://github.com/raghul-parthipan/dont_waste_data

Authors: Raghul Parthipan (University of Cambridge); Damon Wischik (Univeristy of Cambridge)
NeurIPS 2022 Forecasting European Ozone Air Pollution With Transformers (Papers Track)
Abstract and authors: (click to expand)

Abstract: Surface ozone is an air pollutant that contributes to hundreds of thousands of premature deaths annually. Accurate short-term ozone forecasts may allow improved policy to reduce the risk to health, such as air quality warnings. However, forecasting ozone is a difficult problem, as surface ozone concentrations are controlled by a number of physical and chemical processes which act on varying timescales. Accounting for these temporal dependencies appropriately is likely to provide more accurate ozone forecasts. We therefore deploy a state-of-the-art transformer-based model, the Temporal Fusion Transformer, trained on observational station data from three European countries. In four-day test forecasts of daily maximum 8-hour ozone, the novel approach is highly skilful (MAE = 4.6 ppb, R2 = 0.82), and generalises well to two European countries unseen during training (MAE = 4.9 ppb, R2 = 0.79). The model outperforms standard machine learning models on our data, and compares favourably to the published performance of other deep learning architectures tested on different data. We illustrate that the model pays attention to physical variables known to control ozone concentrations, and that the attention mechanism allows the model to use relevant days of past ozone concentrations to make accurate forecasts.

Authors: Seb Hickman (University of Cambridge); Paul Griffiths (University of Cambridge); Alex Archibald (University of Cambridge); Peer Nowack (Imperial College London); Elie Alhajjar (USMA)
NeurIPS 2022 Exploring Randomly Wired Neural Networks for Climate Model Emulation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts, but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this manuscript, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them to their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired ones and assess the impact on model performance for models with 1 million and 10 million parameters. We find average performance improvements of 4.2% across model complexities and prediction tasks, with substantial performance improvements of up to 16.4% in some cases. Furthermore, we find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.

Authors: William J Yik (Harvey Mudd College); Sam J Silva (The University of Southern California); Andrew Geiss (Pacific Northwest National Laboratory); Duncan Watson-Parris (University of Oxford)
NeurIPS 2022 Transformer Neural Networks for Building Load Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate electrical load forecasts of buildings are needed to optimize local energy storage and to make use of demand-side flexibility. We study the usage of Transformer neural networks for short-term electrical load forecasting of 296 buildings from a public dataset. Transformer neural networks trained on many buildings give the best forecasts on 115 buildings, and multi-layer perceptrons trained on a single building are better on 161 buildings. In addition, we evaluate the models on buildings that were not used for training, and find that Transformer neural networks generalize better than multi-layer perceptrons and our statistical baselines. This shows that the usage of Transformer neural networks for building load forecasting could reduce training resources due to the good generalization to unseen buildings, and they could be useful for cold-start scenarios.

Authors: Matthias Hertel (KIT); Simon Ott (KIT); Oliver Neumann (KIT); Benjamin Schäfer (KIT); Ralf Mikut (Karlsruhe Institute of Technology); Veit Hagenmeyer (Karlsruhe Institute of Technology (KIT))
NeurIPS 2022 Heat Demand Forecasting with Multi-Resolutional Representation of Heterogeneous Temporal Ensemble (Papers Track)
Abstract and authors: (click to expand)

Abstract: One of the primal challenges faced by utility companies is ensuring efficient supply with minimal greenhouse gas emissions. The advent of smart meters and smart grids provide an unprecedented advantage in realizing an optimised supply of thermal energies through proactive techniques such as load forecasting. In this paper, we propose a forecasting framework for heat demand based on neural networks where the time series are encoded as scalograms equipped with the capacity of embedding exogenous variables such as weather, and holiday/non-holiday. Subsequently, CNNs are utilized to predict the heat load multi-step ahead. Finally, the proposed framework is compared with other state-of-the-art methods, such as SARIMAX and LSTM. The quantitative results from retrospective experiments show that the proposed framework consistently outperforms the state-of-the-art baseline method with real-world data acquired from Denmark. A minimal mean error of 7.54% for MAPE and 417kW for RMSE is achieved with the proposed framework in comparison to all other methods.

Authors: Satyaki Chatterjee (Pattern Recognition Lab, FAU Erlangen-Nuremberg); Adithya Ramachandran (Pattern Recognition Lab, Friedrich Alexander University, Erlangen); Thorkil Flensmark Neergaard (Brønderslev Forsyning A/S); Andreas K Maier (Pattern Recognition Lab, FAU Erlangen-Nuremberg); Siming Bayer (Pattern Recognition Lab, Friedrich-Alexander University)
NeurIPS 2022 Identifying Compound Climate Drivers of Forest Mortality with β-VAE (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to lead to higher rates of forest mortality. Forest mortality is a complex phenomenon driven by the interaction of multiple climatic variables at multiple temporal scales, further modulated by the current state of the forest (e.g. age, stem diameter, and leaf area index). Identifying the compound climate drivers of forest mortality would greatly improve understanding and projections of future forest mortality risk. Observation data are, however, limited in accuracy and sample size, particularly in regard to forest state variables and mortality events. In contrast, simulations with state-of-the-art forest models enable the exploration of novel machine learning techniques for associating forest mortality with driving climate conditions. Here we simulate 160,000 years of beech, pine and spruce forest dynamics with the forest model FORMIND. We then apply β-VAE to learn disentangled latent representations of weather conditions and identify those that are most likely to cause high forest mortality. The learned model successfully identifies three characteristic climate representations that can be interpreted as different compound drivers of forest mortality.

Authors: Mohit Anand (Helmholtz Centre for Environmental Research - UFZ); Lily-belle Sweet (Helmholtz Centre for Environmental Research - UFZ); Gustau Camps-Valls (Universitat de València); Jakob Zscheischler (Helmholtz Centre for Environmental Research - UFZ)
NeurIPS 2022 Hybrid Recurrent Neural Network for Drought Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Droughts are pervasive hydrometeorological phenomena and global hazards, whose frequency and intensity are expected to increase in the context of climate change. Drought monitoring is of paramount relevance. Here we propose a hybrid model for drought detection that integrates both climatic indices and data-driven models in a hybrid deep learning approach. We exploit time-series of multi-scale Standardized Precipitation Evapotranspiration Index together with precipitation and temperature as inputs. We introduce a dual-branch recurrent neural network with convolutional lateral connections for blending the data. Experimental and ablative results show that the proposed system outperforms both the considered drought index and purely data-driven deep learning models. Our results suggest the potential of hybrid models for drought monitoring and open the door to synergistic systems that learn from data and domain knowledge altogether.

Authors: Mengxue Zhang (Universitat de València); Miguel-Ángel Fernández-Torres (Universitat de València); Gustau Camps-Valls (Universitat de València)
NeurIPS 2022 Comparing the carbon costs and benefits of low-resource solar nowcasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Mitigating emissions in line with climate goals requires the rapid integration of low carbon energy sources, such as solar photovoltaics (PV) into the electricity grid. However, the energy produced from solar PV fluctuates due to clouds obscuring the sun's energy. Solar PV yield nowcasting is used to help anticipate peaks and troughs in demand to support grid integration. This paper compares multiple low-resource approaches to nowcasting solar PV yield. To do so, we use a dataset of UK satellite imagery and solar PV energy readings over a 1 to 4-hour time range. Our work investigates the performance of multiple nowcasting models. The paper also estimates the carbon emissions generated and averted by deploying models such as these, and finds that short-term PV forecasting may have a benefit several orders of magnitude greater than its carbon cost and that this benefit holds for small models that could be deployable in low-resource settings around the globe.

Authors: Ben Dixon (UCL); Jacob Bieker (Open Climate Fix); Maria Perez-Ortiz (University College London)
NeurIPS 2022 Transformers for Fast Emulation of Atmospheric Chemistry Box Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: When modeling atmospheric chemistry, concentrations are determined by numerically solving large systems of ordinary differential equations that represent a set of chemical reactions. These solvers can be very computationally intensive, particularly those with the thousands or tens of thousands of chemical species and reactions that make up the most accurate models. We demonstrate the application of a deep learning transformer architecture to emulate an atmospheric chemistry box model, and show that this attention-based model outperforms LSTM and autoencoder baselines while providing interpretable predictions that are more than 2 orders of magnitude faster than a numerical solver. This work is part of a larger study to replace the numerical solver in a 3D global chemical model with a machine learned emulator and achieve significant speedups for global climate simulations.

Authors: Herbie Bradley (University of Cambridge); Nathan Luke Abraham (National Centre for Atmospheric Science, UK); Peer Nowack (Imperial College London); Doug McNeall (Met Office Hadley Centre, UK)
NeurIPS 2022 Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs (Papers Track)
Abstract and authors: (click to expand)

Abstract: Forecasting the state of vegetation in response to climate and weather events is a major challenge. Its implementation will prove crucial in predicting crop yield, forest damage, or more generally the impact on ecosystems services relevant for socio-economic functioning, which if absent can lead to humanitarian disasters. Vegetation status depends on weather and environmental conditions that modulate complex ecological processes taking place at several timescales. Interactions between vegetation and different environmental drivers express responses at instantaneous but also time-lagged effects, often showing an emerging spatial context at landscape and regional scales. We formulate the land surface forecasting task as a strongly guided video prediction task where the objective is to forecast the vegetation developing at very fine resolution using topography and weather variables to guide the prediction. We use a Convolutional LSTM (ConvLSTM) architecture to address this task and predict changes in the vegetation state in Africa using Sentinel-2 satellite NDVI, having ERA5 weather reanalysis, SMAP satellite measurements, and topography (DEM of SRTMv4.1) as variables to guide the prediction. Ours results highlight how ConvLSTM models can not only forecast the seasonal evolution of NDVI at high resolution, but also the differential impacts of weather anomalies over the baselines. The model is able to predict different vegetation types, even those with very high NDVI variability during target length.

Authors: Claire Robin (Biogeochemical Integration, Max-Planck-Institute for Biogeochemistry, Jena, Germany); Christian Requena-Mesa (Computer Vision Group, Friedrich Schiller University Jena; DLR Institute of Data Science, Jena; Max Planck Institute for Biogeochemistry, Jena); Vitus Benson (Max-Planck-Institute for Biogeochemistry); Jeran Poehls (Max-Planck-Institute for Biogeochemistry); Lazaro Alonzo (Max-Planck-Institute for Biogeochemistry Max-Planck-Institute for Biogeochemistry); Nuno Carvalhais (Max-Planck-Institute for Biogeochemistry); Markus Reichstein (Max Planck Institute for Biogeochemistry, Jena; Michael Stifel Center Jena for Data-Driven and Simulation Science, Jena)
NeurIPS 2022 Identification of medical devices using machine learning on distribution feeder data for informing power outage response (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Power outages caused by extreme weather events due to climate change have doubled in the United States in the last two decades. Outages pose severe health risks to over 4.4 million individuals dependent on in-home medical devices. Data on the number of such individuals residing in a given area is limited. This study proposes a load disaggregation model to predict the number of medical devices behind an electric distribution feeder. This data can be used to inform planning and response to power outages and other emergencies. The proposed solution serves as a measure for climate change adaptation.

Authors: Paraskevi Kourtza (University of Edinburgh); Maitreyee Marathe (University of Wisconsin-Madison); Anuj Shetty (Stanford University); Diego Kiedanski (Yale University)
NeurIPS 2022 Deep-S2SWind: A data-driven approach for improving Sub-seasonal wind predictions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: A major transformation to mitigate climate change implies a rapid decarbonisation of the energy system and thus, increasing the use of renewable energy sources, such as wind power. However, renewable resources are strongly dependent on local and large-scale weather conditions, which might be influenced by climate change. Weather-related risk assessments are essential for the energy sector, in particular, for power system management decisions, for which forecasts of climatic conditions from several weeks to months (i.e. sub-seasonal scales) are of key importance. Here, we propose a data-driven approach to predict wind speed at longer lead-times that can benefit the energy sector. The main goal of this study is to assess the potential of machine learning algorithms to predict periods of low wind speed conditions that have a strong impact on the energy sector.

Authors: Noelia Otero Felipe (University of Bern); Pascal Horton (University of Bern)
NeurIPS 2022 Detecting Floods from Cloudy Scenes: A Fusion Approach Using Sentinel-1 and Sentinel-2 Imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: As the result of climate change, extreme flood events are becoming more frequent. To better respond to such disasters, and to test and calibrate flood models, we need accurate real-world data on flooding extent. Detection of floods from remote sensed imagery suffers from a widespread problem: clouds block flood scenes in images, leading to degraded and fragmented flood datasets. To address this challenge, we propose a workflow based on U-Net, and a dataset that detects flood in cloud-prone areas by fusing information from the Sentinel-1 and Sentinel-2 satellites. The expected result will be a reliable and detailed catalogue of flood extents and how they change through time, allowing us to better understand flooding in different morphological settings and climates.

Authors: Qiuyang Chen (University of Edinburgh); Xenofon Karagiannis (Earth-i Ltd.); Simon M. Mudd (University of Edinburgh)
NeurIPS 2022 Urban Heat Island Detection and Causal Inference Using Convolutional Neural Networks (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Compared to rural areas, urban areas experience higher temperatures for longer periods of time because of the urban heat island (UHI) effect. This increased heat stress leads to greater mortality, increased energy demand, regional changes to precipitation patterns, and increased air pollution. Urban developers can minimize the UHI effect by incorporating features that promote air flow and heat dispersion (e.g., increasing green space). However, understanding which urban features to implement is complex, as local meteorology strongly dictates how the environment responds to changes in urban form. In this proposal we describe a methodology for estimating the causal relationship between changes in urban form and changes in the UHI effect. Changes in urban form and temperature changes are measured using convolutional neural networks, and a causal inference matching approach is proposed to estimate causal relationships. The success of this methodology will enable urban developers to implement city-specific interventions to mitigate the warming planet's impact on cities.

Authors: Zachary D Calhoun (Duke University); Ziyang Jiang (Duke University); Mike Bergin (Duke University); David Carlson (Duke University)
NeurIPS 2022 Interpretable Spatiotemporal Forecasting of Arctic Sea Ice Concentration at Seasonal Lead Times (Proposals Track)
Abstract and authors: (click to expand)

Abstract: There are many benefits from the accurate forecasting of Arctic sea ice, however existing models struggle to reliably predict sea ice concentration at long lead times. Many numerical models exist but can be sensitive to initial conditions, and while recent deep learning-based methods improve overall robustness, they either do not utilize temporal trends or rely on architectures that are not performant at learning long-term sequential dependencies. We propose a method of forecasting sea ice concentration using neural circuit policies, a form of continuous time recurrent neural architecture, which improve the learning of long-term sequential dependencies compared to existing techniques and offer the added benefits of adaptability to irregular sequence intervals and high interpretability.

Authors: Matthew Beveridge (Independent Researcher); Lucas Pereira (ITI, LARSyS, Técnico Lisboa)
NeurIPS 2022 Improving accuracy and convergence of federated learning edge computing methods for generalized DER forecasting applications in power grid (Proposals Track)
Abstract and authors: (click to expand)

Abstract: This proposal aims to develop more accurate federated learning (FL) methods with faster convergence properties and lower communication requirements, specifically for forecasting distributed energy resources (DER) such as renewables, energy storage, and loads in modern, low-carbon power grids. This will be achieved by (i) leveraging recently developed extensions of FL such as hierarchical and iterative clustering to improve performance with non-IID data, (ii) experimenting with different types of FL global models well-suited to time-series data, and (iii) incorporating domain-specific knowledge from power systems to build more general FL frameworks and architectures that can be applied to diverse types of DERs beyond just load forecasting, and with heterogeneous clients.

Authors: Vineet Jagadeesan Nair (MIT)
AAAI FSS 2022 Graph Representation Learning for Energy Demand Data: Application to Joint Energy System Planning under Emissions Constraints
Abstract and authors: (click to expand)

Abstract: To meet the mid-century goal of CO2 emissions reduction requires a rapid transformation of current electric power and natural gas (NG) infrastructure. This necessitates a long-term planning of the joint power-NG system under representative demand patterns, operational constraints, and policy considerations. Our work is motivated by the computational and practical challenges associated with solving the generation and transmission expansion problem (GTEP) for joint planning of power-NG systems. Specifically, we focus on efficiently extracting a set of representative days from power and NG demand data in respective networks and using this set to reduce the computational burden required to solve the GTEP. We propose a Graph Autoencoder for Multiple time resolution Energy Systems (GAMES) to capture the spatio-temporal demand patterns in interdependent networks and account for differences in the temporal resolution of available data. The resulting embeddings are used in a clustering algorithm to select representative days. We evaluate the effectiveness of our approach in solving a GTEP formulation calibrated for the joint power-NG system in New England. This formulation accounts for the physical interdependencies between power and NG systems, including the joint emissions constraint. Our results show that the set of representative days obtained from GAMES not only allows us to tractably solve the GTEP formulation, but also achieves a lower cost of implementing the joint planning decisions.

Authors: Aron Brenner (MIT), Rahman Khorramfar (MIT), Dharik Mallapragada (MIT) and Saurabh Amin (MIT)
AAAI FSS 2022 Self-Supervised Representations of Geo-located Weather Time Series - an Evaluation and Analysis
Abstract and authors: (click to expand)

Abstract: Self-supervised learning (SSL) is gaining traction in various domains, and demonstrated their potential particularly in tasks where labelled data is limited and costly to collect. In this work, we evaluate the performance existing self-supervised multivariate time series learning algorithms on weather-driven applications involving regression, classification and forecasting tasks. We experiment with a two-step protocol. In the first step, we employ and SSL algorithm and learn generic weather representations from multivariate weather data. Then, in the next step, we use these representations and fine-tune them for multiple downstream tasks. Through our initial experiments on air quality prediction and renewable energy generation tasks, we highlight the benefits of self-supervised weather representations, including improved performance in such tasks, ability to generalize with limited in-task data, and reduction in training time and carbon emissions. We expect such a direction to be relevant in multiple weather-driven applications supporting climate change mitigation and adaptation efforts.

Authors: Arjun Ashok (IBM Research), Devyani Lambhate (IBM Research) and Jitendra Singh (IBM Research)
NeurIPS 2021 Towards Representation Learning for Atmospheric Dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: The prediction of future climate scenarios under anthropogenic forcing is critical to understand climate change and to assess the impact of potentially counter-acting technologies. Machine learning and hybrid techniques for this prediction rely on informative metrics that are sensitive to pertinent but often subtle influences. For atmospheric dynamics, a critical part of the climate system, no well established metric exists and visual inspection is currently still often used in practice. However, this ``eyeball metric'' cannot be used for machine learning where an algorithmic description is required. Motivated by the success of intermediate neural network activations as basis for learned metrics, e.g. in computer vision, we present a novel, self-supervised representation learning approach specifically designed for atmospheric dynamics. Our approach, called AtmoDist, trains a neural network on a simple, auxiliary task: predicting the temporal distance between elements of a randomly shuffled sequence of atmospheric fields (e.g. the components of the wind field from reanalysis or simulation). The task forces the network to learn important intrinsic aspects of the data as activations in its layers and from these hence a discriminative metric can be obtained. We demonstrate this by using AtmoDist to define a metric for GAN-based super resolution of vorticity and divergence. Our upscaled data matches both visually and in terms of its statistics a high resolution reference closely and it significantly outperform the state-of-the-art based on mean squared error. Since AtmoDist is unsupervised, only requires a temporal sequence of fields, and uses a simple auxiliary task, it has the potential to be of utility in a wide range of applications.

Authors: Sebastian Hoffmann (University of Magdeburg); Christian Lessig (Otto-von-Guericke-Universitat Magdeburg)
NeurIPS 2021 Addressing Deep Learning Model Uncertainty in Long-Range Climate Forecasting with Late Fusion (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global warming leads to the increase in frequency and intensity of climate extremes that cause tremendous loss of lives and property. Accurate long-range climate prediction allows more time for preparation and disaster risk management for such extreme events. Although machine learning approaches have shown promising results in long-range climate forecasting, the associated model uncertainties may reduce their reliability. To address this issue, we propose a late fusion approach that systematically combines the predictions from multiple models to reduce the expected errors of the fused results. We also propose a network architecture with the novel denormalization layer to gain the benefits of data normalization without actually normalizing the data. The experimental results on long-range 2m temperature forecasting show that the framework outperforms the 30-year climate normals, and the accuracy can be improved by increasing the number of models.

Authors: Ken C. L. Wong (IBM Research – Almaden Research Center); Hongzhi Wang (IBM Almaden Research Center); Etienne E Vos (IBM); Bianca Zadrozny (IBM Research); Campbell D Watson (IBM Reserch); Tanveer Syeda-Mahmood (IBM Research)
NeurIPS 2021 Predicting Discharge in Catchment Outlet Using Deep Learning: Case Study of the Ansongo-Niamey Basin (Papers Track)
Abstract and authors: (click to expand)

Abstract: Hydrological models are one of the key challenges in hydrology. Their goal is to understand, predict and manage water resources. Most of the hydrological models so far were either physical or conceptual models. But in the past two decades, fully data-driven (empirical) models started to emerge with the breakthroughs of novel deep learning methods in runoff prediction. These breakthroughs were mostly favored by the large volume, variety and velocity of water-related data. Long Short-Term Memory and Gated Recurrent Unit neural networks, particularly achieved the outstanding milestone of outperforming classic hydrological models in less than a decade. Moreover, they have the potential to change the way hydrological modeling is performed. In this study, precipitation, minimal and maximum temperature at the Ansongo-Niamey basin combined with the discharge at Ansongo and Kandadji were used to predict the discharge at Niamey using artificial neural networks. After data preprocessing and hyperparameter optimization, the deep learning models performed well with the LSTM and GRU respectively scoring a Nash-Sutcliffe Efficiency of 0.933 and 0.935. This performance matches those of well-known physically-based models used to simulate Niamey’s discharge and therefore demonstrates the efficiency of deep learning methods in a West African context, especially in Niamey which has been facing severe floods due to climate change.

Authors: Peniel J. Y. Adounkpe (WASCAL); Eric Alamou (Université d'Abomey-Calavi); Belko Diallo (WASCAL); Abdou Ali (AGRHYMET Regional Centre)
NeurIPS 2021 Hurricane Forecasting: A Novel Multimodal Machine Learning Framework (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper describes a machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple distinct ML techniques and utilizing diverse data sources. Our framework, which we refer to as Hurricast (HURR), is built upon the combination of distinct data processing techniques using gradient-boosted trees and novel encoder-decoder architectures, including CNN, GRU and Transformers components. We propose a deep-learning feature extractor methodology to mix spatial-temporal data with statistical data efficiently. Our multimodal framework unleashes the potential of making forecasts based on a wide range of data sources, including historical storm data, and visual data such as reanalysis atmospheric images. We evaluate our models with current operational forecasts in North Atlantic (NA) and Eastern Pacific (EP) basins on 2016-2019 for 24-hour lead time, and show our models consistently outperform statistical-dynamical models and compete with the best dynamical models. Furthermore, the inclusion of Hurricast into an operational forecast consensus model leads to a significant improvement of 5% - 15% over NHC's official forecast, thus highlighting the complementary properties with existing approaches.

Authors: Léonard Boussioux (MIT, CentraleSupélec); Cynthia Zeng (MIT); Dimitris Bertsimas (MIT); Théo J Guenais (Harvard University)
NeurIPS 2021 Improved Drought Forecasting Using Surrogate Quantile And Shape (SQUASH) Loss (Papers Track)
Abstract and authors: (click to expand)

Abstract: Droughts are amongst the most damaging natural hazard with cascading impacts across multiple sectors of the economy and society. Improved forecasting of drought conditions ahead of time can significantly improve strategic planning to mitigate the impacts and enhance resilience. Though significant progress in forecasting approaches has been made, the current approaches focus on the overall improvement of the forecast, with less attention on the extremeness of drought events. In this paper, we focus on improving the accuracy of forecasting extreme and severe drought events by introducing a novel loss function Surrogate Quantile and Shape loss (SQUASH) that combines weighted quantile loss and dynamic time-warping-based shape loss. We show the effectiveness of the proposed loss functions for imbalanced time-series drought forecasting tasks on two regions in India and the USA.

Authors: Devyani Lambhate Lambhate (Indian Institute of Science); Smit Marvaniya (IBM Research India); Jitendra Singh (IBM Research - India); David Gold (IBM)
NeurIPS 2021 Subseasonal Solar Power Forecasting via Deep Sequence Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: To help mitigate climate change, power systems need to integrate renewable energy sources, such as solar, at a rapid pace. Widespread integration of solar energy into the power system requires major improvements in solar irradiance forecasting, in order to reduce the uncertainty associated with solar power output. While recent works have addressed short lead-time forecasting (minutes to hours ahead), week(s)-ahead and longer forecasts, coupled with uncertainty estimates, will be extremely important for storage applications in future power systems. In this work, we propose machine learning approaches for these longer lead-times as an important new application area in the energy domain. We demonstrate the potential of several deep sequence learning techniques for both point predictions and probabilistic predictions at these longer lead-times. We compare their performance for subseasonal forecasting (forecast lead-times of roughly two weeks) using the SURFRAD data set for 7 stations across the U.S. in 2018. The results are encouraging; the deep sequence learning methods outperform the current benchmark for machine learning-based probabilistic predictions (previously applied at short lead-times in this domain), along with relevant baselines.

Authors: Saumya Sinha (University of Colorado, Boulder); Bri-Mathias S Hodge (University of Colorado Boulder); Claire Monteleoni (University of Colorado Boulder)
NeurIPS 2021 A GNN-RNN Approach for Harnessing Geospatial and Temporal Information: Application to Crop Yield Prediction (Papers Track) Best Paper: ML Innovation
Abstract and authors: (click to expand)

Abstract: Climate change poses new challenges to agricultural production, as crop yields are extremely sensitive to climatic variation. Accurately predicting the effects of weather patterns on crop yield is crucial for addressing issues such as food insecurity, supply stability, and economic planning. Recently, there have been many attempts to use machine learning models for crop yield prediction. However, these models either restrict their tasks to a relatively small region or a short time-period (e.g. a few years), which makes them hard to generalize spatially and temporally. They also view each location as an i.i.d sample, ignoring spatial correlations in the data. In this paper, we introduce a novel graph-based recurrent neural network for crop yield prediction, which incorporates both geographical and temporal structure. Our method is trained, validated, and tested on over 2000 counties from 41 states in the US mainland, covering years from 1981 to 2019. As far as we know, this is the first machine learning method that embeds geographical knowledge in crop yield prediction and predicts crop yields at the county level nationwide. Experimental results show that our proposed method consistently outperforms a wide variety of existing state-of-the-art methods, validating the effectiveness of geospatial and temporal information.

Authors: Joshua Fan (Cornell University); Junwen Bai (Cornell University); Zhiyun Li (Cornell University); Ariel Ortiz-Bobea (Cornell); Carla P Gomes (Cornell University)
NeurIPS 2021 Prediction of Household-level Heat-Consumption using PSO enhanced SVR Model (Papers Track)
Abstract and authors: (click to expand)

Abstract: In combating climate change, an effective demand-based energy supply operation of the district energy system (DES) for heating or cooling is indispensable. As a consequence, an accurate forecast of heat consumption on the consumer side poses an important first step towards an optimal energy supply. However, due to the non-linearity and non-stationarity of heat consumption data, the prediction of the thermal energy demand of DES remains challenging. In this work, we propose a forecasting framework for thermal energy consumption within a district heating system (DHS) based on kernel Support Vector Regression (kSVR) using real-world smart meter data. Particle Swarm Optimization (PSO) is employed to find the optimal hyper-parameter for the kSVR model which leads to the superiority of the proposed methods when compared to a state-of-the-art ARIMA model. The average MAPE is reduced to 2.07% and 2.64% for the individual meter-specific forecasting and for forecasting of societal consumption, respectively.

Authors: Satyaki Chatterjee (Pattern Recognition Lab, FAU Erlangen-Nuremberg); Siming Bayer (Pattern Recognition Lab, Friedrich-Alexander University); Andreas K Maier (Pattern Recognition Lab, FAU Erlangen-Nuremberg)
NeurIPS 2021 Data Driven Study of Estuary Hypoxia (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper presents a data driven study of dissolved oxygen times series collected in Atlantic Canada. The main motivation of presented work was to evaluate if machine learning techniques could help to understand and anticipate hypoxic episodes in nutrient-impacted estuaries, a phenomenon that is exacerbated by increasing temperature expected to arise due to changes in climate. A major constraint was to limit ourselves to the use of dissolved oxygen time series only. Our preliminary findings shows that recurring neural networks and in particular LSTM may be suitable to predict short horizon levels while traditional results could benefit in longer range hypoxia prevention.

Authors: Md Monwer Hussain (University of New-Brunswick); Guillaume Durand (National Research Council Canada); Michael Coffin (Department of Fisheries and Oceans Canada); Julio J Valdés (National Research Council Canada); Luke Poirier (Department of Fisheries and Oceans Canada)
NeurIPS 2021 High-resolution rainfall-runoff modeling using graph neural network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Time-series modeling has shown great promise in recent studies using the latest deep learning algorithms such as LSTM (Long Short-Term Memory). These studies primarily focused on watershed-scale rainfall-runoff modeling or streamflow forecasting, but the majority of them only considered a single watershed as a unit. Although this simplification is very effective, it does not take into account spatial information, which could result in significant errors in large watersheds. Several studies investigated the use of GNN (Graph Neural Networks) for data integration by decomposing a large watershed into multiple sub-watersheds, but each sub-watershed is still treated as a whole, and the geoinformation contained within the watershed is not fully utilized. In this paper, we propose the GNRRM (Graph Neural Rainfall-Runoff Model), a novel deep learning model that makes full use of spatial information from high-resolution precipitation data, including flow direction and geographic information. When compared to baseline models, GNRRM has less over-fitting and significantly improves model performance. Our findings support the importance of hydrological data in deep learning-based rainfall-runoff modeling, and we encourage researchers to include more domain knowledge in their models.

Authors: Zhongrun Xiang (University of Iowa); Ibrahim Demir (The University of Iowa)
NeurIPS 2021 Machine Learning for Snow Stratigraphy Classification (Papers Track)
Abstract and authors: (click to expand)

Abstract: Snow-layer segmentation and classification is an essential diagnostic task for a wide variety of cryospheric science and climate research applications. To this end a Snow Micro Pen (SMP) can be used - a portable high-resolution snow penetrometer. However, the penetration-force measurements of the SMP must be labeled manually, which is a time-intensive task that requires training and becomes infeasible for large datasets. Here, we evaluate how well machine learning models can automatically segment and classify SMP profiles. Fourteen different models are trained on the MOSAiC SMP dataset, a unique and large SMP dataset of snow on Arctic sea-ice profiles. Depending on the user's task and needs, the long short-term memory neural network and the random forests are performing the best. The findings presented here facilitate and accelerate SMP data analysis and in consequence, help scientists to analyze the effects of climate change on the cryosphere more efficiently.

Authors: Julia Kaltenborn (McGill University); Viviane Clay (Osnabrück University); Amy R. Macfarlane (WSL Institute for Snow and Avalanche Research SLF); Martin Schneebeli (WSL Institute for Snow and Avalanche Research SLF)
NeurIPS 2021 Hybrid physics-based and data-driven modeling with calibrated uncertainty for lithium-ion battery degradation diagnosis and prognosis (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Advancing lithium-ion batteries (LIBs) in both design and usage is key to promoting electrification in the coming decades to mitigate human-caused climate change. Inadequate understanding of LIB degradation is an important bottleneck that limits battery durability and safety. Here, we propose hybrid physics-based and data-driven modeling for online diagnosis and prognosis of battery degradation. Compared to existing battery modeling efforts, we aim to build a model with physics as its backbone and statistical learning techniques as enhancements. Such a hybrid model has better generalizability and interpretability together with a well-calibrated uncertainty associated with its prediction, rendering it more valuable and relevant to safety-critical applications under realistic usage scenarios.

Authors: Jing Lin (Institute for Infocomm Research); Yu Zhang (I2R); Edwin Khoo (Institute for Infocomm Research)
NeurIPS 2021 Machine Learning Speeding Up the Development of Portfolio of New Crop Varieties to Adapt to and Mitigate Climate Change (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate change poses serious challenges to achieving food security in a time of a need to produce more food to keep up with the world’s increasing demand for food. There is an urgent need to speed up the development of new high yielding varieties with traits of adaptation and mitigation to climate change. Mathematical approaches, including ML approaches, have been used to search for such traits, leading to unprecedented results as some of the traits, including heat traits that have been long sought-for, have been found within a short period of time.

Authors: Abdallah Bari (OperAI Canada - Operational AI); Hassan Ouabbou (INRA); Abderrazek Jilal (INRA); Frederick Stoddard (University of Helsinki); Mikko Sillanpää (University of Oulu); Hamid Khazaei (World Vegetable Center)
NeurIPS 2021 DeepQuake: Artificial Intelligence for Earthquake Forecasting Using Fine-Grained Climate Data (Proposals Track) Best Paper: Proposals
Abstract and authors: (click to expand)

Abstract: Earthquakes are one of the most catastrophic natural disasters, making accurate, fine-grained, and real-time earthquake forecasting extremely important for the safety and security of human lives. In this work, we propose DeepQuake, a hybrid physics and deep learning model for fine-grained earthquake forecasting using time-series data of the horizontal displacement of earth’s surface measured from continuously operating Global Positioning System (cGPS) data. Recent studies using cGPS data have established a link between transient deformation within earth's crust to climate variables. DeepQuake’s physics-based pre-processing algorithm extracts relevant features including the x, y, and xy components of strain in earth’s crust, capturing earth’s elastic response to these climate variables, and feeds it into a deep learning neural network to predict key earthquake variables such as the time, location, magnitude, and depth of a future earthquake. Results across California show promising correlations between cGPS derived strain patterns and the earthquake catalog ground truth for a given location and time.

Authors: Yash Narayan (The Nueva School)
NeurIPS 2021 A day in a sustainable life (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: In this notebook, we show the reader how to use an electrical battery to minimize the operational carbon intensity of a building. The central idea is to charge the battery when the carbon intensity of the grid energy mix is low, and vice versa. The same methodology is used in practice to optimise for a number of different objective functions, including energy costs. Taking the hypothetical case of Pi, an eco-conscious and tech-savvy householder in the UK, we walk the reader through getting carbon intensity data, and how to use this with a number of different optimisation algorithms to decarbonise. Starting off with easy-to-understand, brute force search, we establish a baseline for subsequent (hopefully smarter) optimization algorithms. This should come naturally, since in their day job Pi is a data scientist where they often use grid and random search to tune hyperparameters of ML models. The second optimization algorithm we explore is a genetic algorithm, which belongs to the class of derivative free optimizers and is consequently extremely versatile. However, the flexibility of these algorithms comes at the cost of computational speed and effort. In many situations, it makes sense to utilize an optimization method which can make use of the special structure in the problem. As the final step, we see how Pi can optimally solve the problem of minimizing their carbon intensity by formulating it as a linear program. Along the way, we also keep an eye out for some of the most important challenges that arise in practice.

Authors: Hussain Kazmi (KU Leuven); Attila Balint (KU Leuven); Jolien Despeghel (KU Leuven)
ICML 2021 Seasonal Sea Ice Presence Forecasting of Hudson Bay using Seq2Seq Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate and timely forecasts of sea ice conditions are crucial for safe shipping operations in the Canadian Arctic and other ice-infested waters. Given the advancement of machine-learning methods and the recent observations on the declining trend of Arctic sea ice extent over the past decades due to global warming, new machine learning approaches are deployed to provide additional sea ice forecasting products. This study is novel in comparison with previous machine learning (ML) approaches in the sea-ice forecasting domain as it provides a daily spatial map of probability of ice presence in the domain up to 90 days. The predictions are further used to predict freeze-up/breakup dates and show their capability to capture both the variability and the increasing trend of open water season in the domain over the past decades.

Authors: Nazanin Asadi (University of Waterloo); K Andrea Scott (University of Waterloo); Philippe Lamontagne (National Research Council Canada)
ICML 2021 Probabilistic Short-Term Low-Voltage Load Forecasting using Bernstein-Polynomial Normalizing Flows (Papers Track)
Abstract and authors: (click to expand)

Abstract: The transition to a fully renewable energy grid requires better forecasting of demand at the low-voltage level. However, high fluctuations and increasing electrification cause huge forecast errors with traditional point estimates. Probabilistic load forecasts take future uncertainties into account and thus enables various applications in low-carbon energy systems. We propose an approach for flexible conditional density forecasting of short-term load based on Bernstein-Polynomial Normalizing Flows where a neural network controls the parameters of the flow. In an empirical study with 363 smart meter customers, our density predictions compare favorably against Gaussian and Gaussian mixture densities and also outperform a non-parametric approach based on the pinball loss for 24h-ahead load forecasting for two different neural network architectures.

Authors: Marcel Arpogaus (Konstanz University of Applied Sciences); Marcus Voß (Technische Universität Berlin (DAI-Labor)); Beate Sick (ZHAW and University of Zurich); Mark Nigge-Uricher (Bosch.IO GmbH); Oliver Dürr (Konstanz University of Applied Sciences)
ICML 2021 DroughtED: A dataset and methodology for drought forecasting spanning multiple climate zones (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change exacerbates the frequency, duration and extent of extreme weather events such as drought. Previous attempts to forecast drought conditions using machine learning have focused on regional models which have two major limitations for national drought management: (i) they are trained on localised climate data and (ii) their architectures prevent them from being applied to new heterogeneous regions. In this work, we present a new large-scale dataset for training machine learning models to forecast national drought conditions, named DroughtED. The dataset consists of globally available meteorological features widely used for drought prediction, paired with location meta-data which has not previously been utilised for drought forecasting. Here we also establish a baseline on DroughtED and present the first research to apply deep learning models - Long Short-Term Memory (LSTMs) and Transformers - to predict county-level drought conditions across the full extent of the United States. Our results indicate that DroughtED enables deep learning models to learn cross-region patterns in climate data that contribute to drought conditions and models trained on DroughtED compare favourably to state-of-the-art drought prediction models trained on individual regions.

Authors: Christoph D Minixhofer (The University of Edinburgh); Mark Swan (The University of Edinburgh); Calum McMeekin (The University of Edinburgh); Pavlos Andreadis (The University of Edinburgh)
ICML 2021 Forecasting Black Sigatoka Infection Risks with Latent Neural ODEs (Papers Track)
Abstract and authors: (click to expand)

Abstract: Black Sigatoka is the most widely-distributed and destructive disease affecting banana plants. Due to the heavy financial burden of managing this infectious disease, farmers in developing countries face significant banana crop losses. The spread of black Sigatoka is highly dependent on weather conditions and though scientists have produced mathematical models of infectious diseases, adapting these models to incorporate climate effects is difficult. We present MR. NODE (Multiple predictoR Neural ODE), a neural network that models the dynamics of black Sigatoka infection learnt directly from data via Neural Ordinary Differential Equations. Our method encodes external predictor factors into the latent space in addition to the variable that we infer, and it can also predict the infection risk at an arbitrary point in time. Empirically, we demonstrate on historical climate data that our method has superior generalization performance on time points up to one month in the future and unseen irregularities. We believe that our method can be a useful tool to control the spread of black Sigatoka.

Authors: Yuchen Wang (University of Toronto); Matthieu Chan Chee (University of Toronto); Ziyad Edher (University of Toronto); Minh Duc Hoang (University of Toronto); Shion Fujimori (University of Toronto); Jesse Bettencourt (University of Toronto)
ICML 2021 Decadal Forecasts with ResDMD: a residual DMD neural network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Significant investment is being made by operational forecasting centers to produce decadal (1-10 year) forecasts that can support long-term decision making for a more climate-resilient society. One method that has been employed for this task is the Dynamic Mode Decomposition (DMD) algorithm – also known as the Linear Inverse Model– which is used to fit linear dynamical models to data. While the DMD usually approximates non-linear terms in the true dynamics as a linear system with random noise, we investigate an extension to the DMD to explicitly represent the non-linear terms as a neural network. Our weight initialization allows the network to produce sensible results before training and then improve the prediction after training as data becomes available. In this short paper, we evaluate the proposed architecture for simulating global sea surface temperatures and compare the results with the standard DMD and seasonal forecasts produced by the state-of-the-art dynamical model, CFSv2.

Authors: EDUARDO ROCHA RODRIGUES (IBM Research); Campbell Watson (IBM Reserch); Bianca Zadrozny (IBM Research); David Gold (IBM Global Business Services)
ICML 2021 Fast-Slow Streamflow Model Using Mass-Conserving LSTM (Papers Track)
Abstract and authors: (click to expand)

Abstract: Streamflow forecasting is key to effectively managing water resources and preparing for the occurrence of natural calamities being exacerbated by climate change. Here we use the concept of fast and slow flow components to create a new mass-conserving Long Short-Term Memory (LSTM) neural network model. It uses hydrometeorological time series and catchment attributes to predict daily river discharges. Preliminary results evidence improvement in skills for different scores compared to the recent literature.

Authors: Miguel Paredes Quinones (IBM Research); Maciel Zortea (IBM Research); Leonardo Martins (IBM Research)
ICML 2021 Extreme Precipitation Seasonal Forecast Using a Transformer Neural Network (Papers Track)
Abstract and authors: (click to expand)

Abstract: An impact of climate change is the increase in frequency and intensity of extreme precipitation events. However, confidently predicting the likelihood of extreme precipitation at seasonal scales remains an outstanding challenge. Here, we present an approach to forecasting the quantiles of the maximum daily precipitation in each week up to six months ahead using the temporal fusion transformer (TFT) model. Through experiments in two regions, we compare TFT predictions with those of two baselines: climatology and a calibrated ECMWF SEAS5 ensemble forecast (S5). Our results show that, in terms of quantile risk at six month lead time, the TFT predictions significantly outperform those from S5 and show an overall small improvement compared to climatology. The TFT also responds positively to departures from normal that climatology cannot.

Authors: Daniel Salles Civitarese (IBM Research, Brazil); Daniela Szwarcman (IBM Research); Bianca Zadrozny (IBM Research); Campbell Watson (IBM Reserch)
ICML 2021 Self-Supervised Learning on Multispectral Satellite Data for Near-Term Solar Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: With the unprecedented increase in distributed photovoltaic (PV) capacity across the globe, there is an increasing need for reliable and accurate forecasting of solar power generation. While PV output is affected by many factors, the atmosphere, i.e., cloud cover, plays a dominant role in determining the amount of downwelling solar irradiance that reaches PV modules. This paper demonstrates that self-supervised learning of multispectral satellite data from the recently launched GOES-R series of satellites can improve near-term (15 minutes) solar forecasting. We develop deep auto-regressive models using convolutional neural networks (CNN) and long short-term memory networks (LSTM) that are globally trained across many solar sites on the raw spatio-temporal data from GOES-R satellites. This self-supervised model provides estimates of future solar irradiance that can be fed directly to a regression model trained on smaller site-specific solar data to provide near-term solar PV forecasts at the site. The regression implicitly models site-specific characteristics, such as capacity, panel tilt, orientation, etc, while the self-supervised CNN-LSTM implicitly captures global atmospheric patterns affecting a site's solar irradiation. Results across 25 solar sites show the utility of such self-supervised modeling by providing accurate near-term forecast with errors close to that of a model using current ground-truth observations.

Authors: Akansha Singh Bansal (University of Massachusetts Amherst); Trapit Bansal (University of Massachusetts Amherst); David Irwin (University of Massachusetts Amherst)
ICML 2021 Forecasting Sea Ice Concentrations using Attention-based Ensemble LSTM (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurately forecasting Arctic sea ice from sub-seasonal to seasonal scales has been a major scientific effort with fundamental challenges at play. In addition to physics-based earth system models, researchers have been applying multiple statistical and machine learning models for sea ice forecasting. Looking at the potential of data-driven sea ice forecasting, we propose an attention-based Long Short Term Memory (LSTM) ensemble method to predict monthly sea ice extent up to 1 month ahead. Using daily and monthly satellite retrieved sea ice data from NSIDC and atmospheric and oceanic variables from ERA5 reanalysis product for 39 years, we show that our multi-temporal ensemble method outperforms several baseline and recently proposed deep learning models. This will substantially improve our ability in predicting future Arctic sea ice changes, which is fundamental for forecasting transporting routes, resource development, coastal erosion, threats to Arctic coastal communities and wildlife.

Authors: Sahara Ali (University of Maryland, Baltimore County); Yiyi Huang (University of Maryland, Baltimore County); Xin Huang (University of Maryland, Baltimore County); Jianwu Wang (University of Maryland, Baltimore County)
ICML 2021 A study of battery SoC scheduling using machine learning with renewable sources (Papers Track)
Abstract and authors: (click to expand)

Abstract: An open energy system (OES) enables the shared distribution of energy resources within a community autonomously and efficiently. For this distributed system a rooftop solar panel and a battery are installed in each house of the community. The OES system monitors the State of Charge (SoC) of each battery independently, arbitrates energy-exchange requests from each house, and physically controls peer-to-peer energy exchanges. In this study, our goal is to optimize those energy exchanges to maximize the renewable energy penetration within the community using machine learning techniques. Future household electricity consumption is predicted using machine learning from the past time series. The predicted consumption is used to determine the next energy-exchange strategy, i.e. when and how much energy should be exchanged to minimize the surplus of solar energy. The simulation results show that the proposed method can increase the amount of renewable energy penetration within the community.

Authors: Daisuke Kawamoto (Sony Computer Science Laboratories, Inc.); Gopinath Rajendiran (CSIR Central Scientific Instruments Organisation, Chennai)
ICML 2021 Short-term Hourly Streamflow Prediction with Graph Convolutional GRU Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: The frequency and impact of floods are expected to increase due to climate change. It is crucial to predict streamflow, consequently flooding, in order to prepare and mitigate its consequences in terms of property damage and fatalities. This paper presents a Graph Convolutional GRUs based model to predict the next 36 hours of streamflow for a sensor location using the upstream river network. As shown in experiment results, the model presented in this study provides better performance than the persistence baseline and a Convolutional Bidirectional GRU network for the selected study area in short-term streamflow prediction.

Authors: Muhammed A Sit (The University of Iowa); Bekir Demiray (The University of Iowa); Ibrahim Demir (The University of Iowa)
ICML 2021 Deep Spatial Temporal Forecasting of Electrical Vehicle Charging Demand (Papers Track)
Abstract and authors: (click to expand)

Abstract: Electric vehicles can offer a low carbon emission solution to reverse rising emission trends. However, this requires that the energy used to meet the demand is green. To meet this requirement, accurate forecasting of the charging demand is vital. Short and long-term charging demand forecasting will allow for better optimisation of the power grid and future infrastructure expansions. In this paper, we propose to use publicly available data to forecast the electric vehicle charging demand. To model the complex spatial-temporal correlations between charging stations, we argue that Temporal Graph Convolution Models are the most suitable to capture the correlations. The proposed Temporal Graph Convolutional Networks provide the most accurate forecasts for short and long-term forecasting compared with other forecasting methods.

Authors: Frederik B Hüttel (Technical University of Denmark (DTU)); Filipe Rodrigues (Technical University of Denmark (DTU)); Inon Peled (Technical University of Denmark (DTU)); Francisco Pereira (DTU)
ICML 2021 Solar PV Maps for Estimation and Forecasting of Distributed Solar Generation (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Rapid uptake of distributed solar PV is starting to make the operation of grids and energy markets more challenging, and better methods are needed for measuring and forecasting distributed solar PV generation across entire regions. We propose a method for converting time series data from a number of point sources (power measurements at individual sites) into 2-dimensional maps that estimate total solar PV generation across large areas. These maps may be used on their own, or in conjunction with additional data sources (such as satellite imagery) in a deep learning framework that enables improved regional solar PV estimation and forecasting. We provide some early validation and results, discuss anticipated benefits of this approach, and argue that this method has the potential to further enable significant uptake of solar PV, assisting a shift away from fossil fuel-based generation.

Authors: Julian de Hoog (The University of Melbourne); Maneesha Perera (The University of Melbourne); Kasun Bandara (The University of Melbourne); Damith Senanayake (The University of Melbourne); Saman Halgamuge (University of Melbourne)
ICML 2021 Leveraging Machine Learning for Equitable Transition of Energy Systems (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Our planet is facing overlapping crises of climate change, global pandemic, and systemic inequality. To respond climate change, the energy system is in the midst of its most foundational transition since its inception, from traditional fuel-based energy sources to clean renewable sources. While the transition to a low-carbon energy system is in progress, there is an opportunity to make the new system more just and equitable than the current one that is inequitable in many forms. Measuring inequity in the energy system is a formidable task since it is large scale and the data is coming from abundant data sources. In this work, we lay out a plan to leverage and develop scalable machine learning (ML) tools to measure the equity of the current energy system and to facilitate a just transition to a clean energy system. We focus on two concrete examples. First, we explore how ML can help to measure the inequity in the energy inefficiency of residential houses in the scale of a town or a country. Second, we explore how deep learning techniques can help to estimate the solar potential of residential buildings to facilitate a just installation and incentive allocation of solar panels. The application of ML for energy equity is much broader than the above two examples and we highlight some others as well. The result of this research could be used by policymakers to efficiently allocate energy assistance subsidies in the current energy systems and to ensure justice in their energy transition plans.

Authors: Enea Dodi (UMass Amherst); Anupama A Sitaraman (University of Massachusetts Amherst); Mohammad Hajiesmaili (UMass Amherst); Prashant Shenoy (University of Massachusetts Amherst)
ICML 2021 Forecasting emissions through Kaya identity using Neural ODEs (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Starting from Kaya identity, we used a Neural ODE model to predict the evolution of several indicators related to carbon emissions, on a country-level : population, GDP per capita, energy intensity of GDP, carbon intensity of energy. We compared the model with a baseline statistical model - VAR - and obtained good performances. We conclude that this machine-learning approach can be used to produce a wide range of results and give relevant insight to policymakers.

Authors: Pierre Browne (Imperial College London)
ICML 2021 Reducing greenhouse gas emissions by optimizing room temperature set-points (Proposals Track)
Abstract and authors: (click to expand)

Abstract: We design a learning and optimization framework to mitigate greenhouse gas emissions associated with heating and cooling buildings. The framework optimizes room temperature set-points based on forecasts of weather, occupancy, and the greenhouse gas intensity of electricity. We compare two approaches: the first one combines a linear load forecasting model with convex optimization that offers a globally optimal solution, whereas the second one combines a nonlinear load forecasting model with nonconvex optimization that offers a locally optimal solution. The project explores the two approaches with a simulation testbed in EnergyPlus and experiments in university-campus buildings.

Authors: Yuan Cai (MIT); Subhro Das (MIT-IBM Watson AI Lab, IBM Research); Leslie Norford (Massachusetts Institute of Technology); Jeremy Gregory (Massachusetts Institute of Technology); Julia Wang (Massachusetts Institute of Technology); Kevin J Kircher (MIT); Jasmina Burek (Massachusetts Institute of Technology)
ICML 2021 APPLYING TRANSFORMER TO IMPUTATION OF MULTI-VARIATE ENERGY TIME SERIES DATA (Proposals Track)
Abstract and authors: (click to expand)

Abstract: To reduce the greenhouse gas emissions from electricity production, it is necessaryto switch to an energy system based on renewable energy sources (RES). However,intermittent electricity generation from RES poses challenges for energy systems.The primary input for data-driven solutions is data on electricity generation fromRES, which usually contain many missing values. This proposal studies the useof attention-based algorithms to impute missing values of electricity production,electricity demand and electricity prices. Since attention mechanisms allow us totake into account dependencies between time series across multiple dimensionsefficiently, our approach goes beyond classic statistical methods and incorporatesmany related variables, such as electricity price, demand and production by othersources. Our preliminary results show that while transformers can come at highercomputational costs, they are more precise than classical imputation methods.

Authors: Hasan Ümitcan Yilmaz (Karlsruhe Institute of Technology); Max Kleinebrahm (Karlsruhe Institut für Technologie); Christopher Bülte (Karlsruhe Institute of Technology); Juan Gómez-Romero (Universidad de Granada)
NeurIPS 2020 A Temporally Consistent Image-based Sun Tracking Algorithm for Solar Energy Forecasting Applications (Papers Track)
Abstract and authors: (click to expand)

Abstract: Improving irradiance forecasting is critical to further increase the share of solar in the energy mix. On a short time scale, fish-eye cameras on the ground are used to capture cloud displacements causing the local variability of the electricity production. As most of the solar radiation comes directly from the Sun, current forecasting approaches use its position in the image as a reference to interpret the cloud cover dynamics. However, existing Sun tracking methods rely on external data and a calibration of the camera, which requires access to the device. To address these limitations, this study introduces an image-based Sun tracking algorithm to localise the Sun in the image when it is visible and interpolate its daily trajectory from past observations. We validate the method on a set of sky images collected over a year at SIRTA's lab. Experimental results show that the proposed method provides robust smooth Sun trajectories with a mean absolute error below 1% of the image size.

Authors: Quentin Paletta (University of Cambridge); Joan Lasenby (University of Cambridge)
NeurIPS 2020 Meta-modeling strategy for data-driven forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurately forecasting the weather is a key requirement for climate change mitigation. Data-driven methods offer the ability to make more accurate forecasts, but lack interpretability and can be expensive to train and deploy if models are not carefully developed. Here, we make use of two historical climate data sets and tools from machine learning, to accurately predict temperature fields. Furthermore, we are able to use low fidelity models that are cheap to train and evaluate, to selectively avoid expensive high fidelity function evaluations, as well as uncover seasonal variations in predictive power. This allows for an adaptive training strategy for computationally efficient geophysical emulation.

Authors: Dominic J Skinner (MIT); Romit Maulik (Argonne National Laboratory)
NeurIPS 2020 Using attention to model long-term dependencies in occupancy behavior (Papers Track)
Abstract and authors: (click to expand)

Abstract: Over the past years, more and more models have been published that aim to capture relationships in human residential behavior. Most of these models are different Markov variants or regression models that have a strong assumption bias and are therefore unable to capture complex long-term dependencies and the diversity in occupant behavior. This work shows that attention based models are able to capture complex long-term dependencies in occupancy behavior and at the same time adequately depict the diversity in behavior across the entire population and different socio-demographic groups. By combining an autoregressive generative model with an imputation model, the advantages of two data sets are combined and new data are generated which are beneficial for multiple use cases (e.g. generation of consistent household energy demand profiles). The two step approach generates synthetic activity schedules that have similar statistical properties as the empirical collected schedules and do not contain direct information about single individuals. Therefore, the presented approach forms the basis to make data on occupant behavior freely available, so that further investigations based on the synthetic data can be carried out without a large data application effort. In future work it is planned to take interpersonal dependencies into account in order to be able to generate entire household behavior profiles.

Authors: Max Kleinebrahm (Karlsruhe Institut für Technologie); Jacopo Torriti (University Reading); Russell McKenna (University of Aberdeen); Armin Ardone (Karlsruhe Institut für Technologie); Wolf Fichtner (Karlsruhe Institute of Technology)
NeurIPS 2020 Movement Tracks for the Automatic Detection of Fish Behavior in Videos (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global warming is predicted to profoundly impact ocean ecosystems. Fish behavior is an important indicator of changes in such marine environments. Thus, the automatic identification of key fish behavior in videos represents a much needed tool for marine researchers, enabling them to study climate change-related phenomena. We offer a dataset of sablefish (Anoplopoma fimbria) startle behaviors in underwater videos, and investigate the use of deep learning (DL) methods for behavior detection on it. Our proposed detection system identifies fish instances using DL-based frameworks, determines trajectory tracks, derives novel behavior-specific features, and employs Long Short-Term Memory (LSTM) networks to identify startle behavior in sablefish. Its performance is studied by comparing it with a state-of-the-art DL-based video event detector.

Authors: Declan GD McIntosh (University Of Victoria); Tunai Porto Marques (University of Victoria); Alexandra Branzan Albu (University of Victoria); Rodney Rountree (University of Victoria); Fabio De Leo Cabrera (Ocean Networks Canada)
NeurIPS 2020 FlowDB: A new large scale river flow, flash flood, and precipitation dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: Flooding results in 8 billion dollars of damage annually in the US and causes the most deaths of any weather related event. Due to climate change scientists expect more heavy precipitation events in the future. However, no current datasets exist that contain both hourly precipitation and river flow data. We introduce a novel hourly river flow and precipitation dataset and a second subset of flash flood events with damage estimates and injury counts. Using these datasets we create two challenges (1) general stream flow forecasting and (2) flash flood damage estimation. We also create a public benchmark and an Python package to enable easy adding of new models . Additionally, in the future we aim to augment our dataset with snow pack data and soil index moisture data to improve predictions

Authors: Isaac Godfried (CoronaWhy)
NeurIPS 2020 A Comparison of Data-Driven Models for Predicting Stream Water Temperature (Papers Track)
Abstract and authors: (click to expand)

Abstract: Changes to the Earth's climate are expected to negatively impact water resources in the future. It is important to have accurate modelling of river flow and water quality to make optimal decisions for water management. Machine learning and deep learning models have become promising methods for making such hydrological predictions. Using these models, however, requires careful consideration both of data constraints and of model complexity for a given problem. Here, we use machine learning (ML) models to predict monthly stream water temperature records at three monitoring locations in the Northwestern United States with long-term datasets, using meteorological data as predictors. We fit three ML models: a Multiple Linear Regression, a Random Forest Regression, and a Support Vector Regression, and compare them against two baseline models: a persistence model and historical model. We show that all three ML models are reasonably able to predict mean monthly stream temperatures with root mean-squared errors (RMSE) ranging from 0.63-0.91 degrees Celsius. Of the three ML models, Support Vector Regression performs the best with an error of 0.63-0.75 degrees Celsius. However, all models perform poorly on extreme values of water temperature. We identify the need for machine learning approaches for predicting extreme values for variables such as water temperature, since it has significant implications for stream ecosystems and biota.

Authors: Helen Weierbach (Lawrence Berkeley); Aranildo Lima (Aquatic Informatics); Danielle Christianson (Lawrence Berkeley National Lab); Boris Faybishenko (Lawrence Berkeley National Lab); Val Hendrix (Lawrence Berkeley National Lab); Charuleka Varadharajan (Lawrence Berkeley National Lab)
NeurIPS 2020 Automated Salmonid Counting in Sonar Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: The prosperity of salmonids is crucial for several ecological and economic functions. Accurately counting spawning salmonids during their seasonal migration is essential in monitoring threatened populations, assessing the efficacy of recovery strategies, guiding fishing season regulations, and supporting the management of commercial and recreational fisheries. While several different methods exist for counting river fish, they all rely heavily on human involvement, introducing a hefty financial and time burden. In this paper we present an automated fish counting method that utilizes data captured from ARIS sonar cameras to detect and track salmonids migrating in rivers. Our results show that our fully automated system has a 19.3% per-clip error when compared to human counting performance. There is room to improve, but our system can already decrease the amount of time field biologists and fishery managers need to spend manually watching ARIS clips.

Authors: Peter Kulits (Caltech); Angelina Pan (Caltech); Sara M Beery (Caltech); Erik Young (Trout Unlimited); Pietro Perona (California Institute of Technology); Grant Van Horn (Cornell University)
NeurIPS 2020 OfficeLearn: An OpenAI Gym Environment for Building Level Energy Demand Response (Papers Track)
Abstract and authors: (click to expand)

Abstract: Energy Demand Response (DR) will play a crucial role in balancing renewable energy generation with demand as grids decarbonize. There is growing interest in developing Reinforcement Learning (RL) techniques to optimize DR pricing, as pricing set by electric utilities often cannot take behavioral irrationality into account. However, so far, attempts to standardize RL efforts in this area do not exist. In this paper, we present a first of the kind OpenAI gym environment for testing DR with occupant level building dynamics. We demonstrate the variety of parameters built into our office environment allowing the researcher to customize a building to meet their specifications of interest. We hope that this work enables future work in DR in buildings.

Authors: Lucas Spangher (U.C. Berkeley); Akash Gokul (University of California at Berkeley); Utkarsha Agwan (U.C. Berkeley); Joseph Palakapilly (UC Berkeley); Manan Khattar (University of California at Berkeley); Akaash Tawade (University of California at Berkeley); Costas J. Spanos (University of California at Berkeley)
NeurIPS 2020 Privacy Preserving Demand Forecasting to Encourage Consumer Acceptance of Smart Energy Meters (Proposals Track)
Abstract and authors: (click to expand)

Abstract: In this proposal paper we highlight the need for privacy preserving energy demand forecasting to allay a major concern consumers have about smart meter installations. High resolution smart meter data can expose many private aspects of a consumer’s household such as occupancy, habits and individual appliance usage. Yet smart metering infrastructure has the potential to vastly reduce carbon emissions from the energy sector through improved operating efficiencies. We propose the application of a distributed machine learning setting known as federated learning for energy demand forecasting at various scales to make load prediction possible whilst retaining the privacy of consumers’ raw energy consumption data.

Authors: Christopher Briggs (Keele University); Zhong Fan (Keele University); Peter Andras (Keele University, School of Computing and Mathematics, Newcastle-under-Lyme, UK)
NeurIPS 2020 Graph Neural Networks for Improved El Niño Forecasting (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns called teleconnections. Hence, we propose the application of spatiotemporal Graph Neural Networks (GNN) to forecast ENSO at long lead times, finer granularity and improved predictive skill than current state-of-the-art methods. The explicit modeling of information flow via edges may also allow for more interpretable forecasts. Preliminary results are promising and outperform state-of-the art systems for projections 1 and 3 months ahead.

Authors: Salva Rühling Cachay (Technical University of Darmstadt); Emma Erickson (University of Illinois at Urbana-Champaign); Arthur F C Bucker (University of São Paulo); Ernest J Pokropek (Warsaw University of Techology); Willa Potosnak (Duquesne University); Salomey Osei (African Master's of Machine Intelligence(AMMI-GH)); Björn Lütjens (MIT)
NeurIPS 2020 A Multi-source, End-to-End Solution for Tracking Climate Change Adaptation in Agriculture (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The impact of climate change on tropical agri-food systems will depend on both the direction and magnitude of climate change, and the agricultural sector’s adaptive capacity, the latter being affected by the chosen adaptation strategies. By extending SEIRS, a Satellite Remote Sensing (SRS) based system originally developed by the International Center for Tropical Agriculture - CIAT for monitoring U.S. Government-funded development programs across cropping areas in Africa, this research proposes the development and deployment of a scalable AI-based platform exploiting free-of-charge SRS data that will enable the agri-food sector to monitor a wide range of climate change adaptation (CCA) interventions in a timely, evidence-driven and comparable manner. The main contributions of the platform are i) ingesting and processing variety sources of SRS data with a considerable record (> 5 years) of vegetation greenness and precipitation (input data); ii) operating an end-to-end system by exploiting AI-based models suited to time series analysis such as Seq2Seq and Transformers; iii) providing customised proxies informing the success or failure of a given local CCA intervention(s).

Authors: Alejandro Coca-Castro (Kings College London); Aaron Golden (NUI Galway); Louis Reymondin (The Alliance of Bioversity International and the International Center for Tropical Agriculture)
NeurIPS 2020 Forecasting Marginal Emissions Factors in PJM (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Many climate change applications rely on accurate forecasts of power grid emissions, but many forecasting methods can be expensive, sensitive to input errors, or lacking in domain knowledge. Motivated by initial experiments using deep learning and power system modeling techniques, we propose a method that combines the strengths of both of these approaches to forecast hourly day-ahead MEFs for the PJM region of the United States.

Authors: Amy H Wang (Western University); Priya L Donti (Carnegie Mellon University)
NeurIPS 2020 Artificial Intelligence, Machine Learning and Modeling for Understanding the Oceans and Climate Change (Proposals Track)
Abstract and authors: (click to expand)

Abstract: These changes will have a drastic impact on almost all forms of life in the ocean with further consequences on food security, ecosystem services in coastal and inland communities. Despite these impacts, scientific data and infrastructures are still lacking to understand and quantify the consequences of these perturbations on the marine ecosystem. Understanding this phenomenon is not only an urgent but also a scientifically demanding task. Consequently, it is a problem that must be addressed with a scientific cohort approach, where multi-disciplinary teams collaborate to bring the best of different scientific areas. In this proposal paper, we describe our newly launched four-years project focused on developing new artificial intelligence, machine learning, and mathematical modeling tools to contribute to the understanding of the structure, functioning, and underlying mechanisms and dynamics of the global ocean symbiome and its relation with climate change. These actions should enable the understanding of our oceans and predict and mitigate the consequences of climate change.

Authors: Nayat Sánchez Pi (Inria); Luis Martí (Inria); André Abreu (Fountation Tara Océans); Olivier Bernard (Inria); Colomban de Vargas (CNRS); Damien Eveillard (Univ. Nantes); Alejandro Maass (CMM, U. Chile); Pablo Marquet (PUC); Jacques Sainte-Marie (Inria); Julien Salomin (Inria); Marc Schoenauer (INRIA); Michele Sebag (LRI, CNRS, France)