Performance Benchmarking of Data Augmentation and GPU Count Variability for Deep Learning Tornado Predictions

Abstract

Predicting violent storms and dangerous weather conditions with current mod- els can take a long time due to the immense complexity associated with weather

simulation. Machine learning has the potential to classify tornadic weather

patterns much more rapidly, thus allowing for more timely alerts to the pub- lic. To deal with class imbalance challenges in machine learning, different data

augmentation approaches have been proposed. In this work, we examine the wall time difference between live data augmentation methods versus the use of preaugmented data when they are used in a convolutional neural network based training for tornado prediction. We also compare CPU and GPU based training over varying sizes of augmented data sets. Additionally we examine what impact varying the number of GPUs used for training will produce given a convolutional neural network on wall time and accuracy. We conclude that using multiple GPUs to train a single network has no signficant advantage over using a single GPU. The number of GPUs used during training should be kept as small as possible for maximum search throughput as the native keras multi-GPU model provides little speedup with optimal learning parameters.