Supervised and Semi-Supervised Semantic Segmentation For Natural Disaster Damage Assessment

Abstract

Natural disasters cause personal injuries, and financial hardship for the whole society. Due to climate change the incidents of natural disasters are increasing with higher intensity. During and even immediately after a natural disaster an assessment of the impacted area is crucial to the rescue team to provide the necessary and immediate help. We can utilize the recent advancement in the field of computer vision to extract essential information in this scenario and provide important help in rescue efforts. Despite the urgency and the importance of application of computer vision techniques in this field there is not much research and resources available. This thesis focuses on the developing datasets and methodologies for natural disaster damage assessment. This thesis presents two datasets with pixel-level annotation named FloodNet and RescueNet. The images of FloodNet were collected after hurricane Harvey while RescueNet is built upon the images collected after hurricane Michael. Before the creation of RescueNet, an initial version was released as HRUD. FloodNet consists of 9 classes and RescueNet includes 10 classes including 4 different building damage classes. We also developed two self-attention based semantic segmentation methods and implemented them on the developed datasets. The developed methods have shown superior performance on these two datasets compared to other state-of-the-art semantic segmentation methods. However, the developed supervised segmentation models are heavier in size and therefore not suitable for edge device implementation for real-world application. In future, we intend to reduce the model complexity and design more lighter models for ease of edge device implementation. Despite higher accuracy attained by the developed methods, scarcity of labeled images in the datasets is a roadblock for developing deep learning methods. Recent deep learning methods require a lot of data to train the models. Semi-supervised learning can be utilized in this scenario. Although semi-supervised learning is getting popular in the computer vision community, there has not been much work done in terms of semantic segmentation. In this thesis, we develop a semi-supervised semantic segmentation method implementing consistency regularization. The detailed experimental analysis shows that our developed semi-supervised performed better existing state-of-the-art semi-supervised methods and can further improve the accuracy of segmentation using unlabeled data. Despite improvement upon supervised methods, semi-supervised methods still suffer from distribution between the labeled and unlabeled data while applying unlabeled data in the training process and degrade the results. In future, we intend to address the distribution shift and class imbalance in the dataset during the model development.