Visual Question Answering: A Deep Interactive Framework for Post-Disaster Management and Damage Assessment (Papers Track)
Argho Sarkar (University of Maryland, Baltimore County); Maryam Rahnemoonfar (University of Maryland Baltimore County)
Abstract
Each natural disaster has left a trail of destruction and damage, which must be managed very effectively to reduce the disaster's impact. Lack of proper decision making in post-disaster managerial level can increase human suffering and waste a great amount of money. Our objective is to incorporate a deep interactive approach in the decision-making system especially in a rescue mission after any natural disaster for the systematic distribution of the limited resources and accelerating the recovery process. We believe that Visual Question Answering (VQA) is the finest way to address this issue. In visual question answering, a query-based answer regarding the situation of the affected area can add value to the decision-making system. Our main purpose of this study is to develop a Visual Question Answering model for post-disaster damage assessment purposes. With this aim, we collect the images by UAV (Unmanned Aerial Vehicle) after Hurricane Harvey and develop a dataset that includes the questions that are very important in the decision support system after a natural disaster. In addition, We propose a supervised attention-based approach in the modeling segment. We compare our approach with the two other baseline attention-based VQA algorithms namely Multimodal Factorized Bilinear (MFB) and Stacked Attention Network (SAN). Our approach outperforms in providing answers for several types of queries including simple counting, complex counting compares to the baseline models.