Evaluation of NAQFC model performance in forecasting surface ozone during the 2011 DISCOVER-AQ campaign

Abstract

The National Air Quality Forecast Capability (NAQFC) and an experimental version of the NAQFC (NAQFC-β) provided flight decision support during the July 2011 NASA DISCOVER-AQ field campaign around Baltimore, Maryland. Ozone forecasts from the NAQFC and NAQFC-β were compared to surface observations at six air quality monitoring stations in the DISCOVER-AQ domain. A bootstrap algorithm was used to test for significant bias and error in the forecasts from each model. Both models produce significant positively biased forecasts in the morning while generally becoming insignificantly biased in the afternoon during peak ozone hours. The NAQFC-β produces higher forecast bias, higher forecast error, and lower correlations than the NAQFC. Forecasts from the two models were also compared to each other to determine the spatial and temporal extent of significant differences in forecasted ozone using a bootstrap algorithm. The NAQFC-β tends to produce an average background ozone mixing ratio of at least 3.51 ppbv greater than the NAQFC throughout the domain at 95 % significance. The difference between the two models is significant during the overnight and early morning hours likely due to the way the Carbon Bond 5 mechanism in the NAQFC-β handles reactive nitrogen recycling and organic peroxide species. The value of information each model provides was tested using a static cost-loss ratio model. By standard measures of forecast skill, the NAQFC generally outperforms the NAQFC-β; however, the NAQFC-β provides greater value of information. This is because standard measures of forecast skill often hide the sensitivity of end users’ needs to forecast error.