Evaluating Machine Learning-Enhanced Sub-Grid Scale Stress Models With Invariance Embedding for Meso-Scale Hurricane Boundary Layer Flows

Abstract

Abstract. This study evaluates several machine learning-based subgrid scale (SGS) stress models for large-eddy simulations (LES) of mesoscale hurricane boundary layer flows. Building on previous work that directly modeled the Smagorinsky constant (Cₛ), we focus on predicting the deviatoric part of the SGS stress tensor (τ*) using feed-forward neural networks here. Two optimization strategies are compared: MATLAB’s Bayesian Regularization (BR), which provides strong generalization but is computationally expensive, and PyTorch’s ADAM optimizer, which offers better scalability with GPU acceleration. Our results show that Bayesian regularization achieves higher accuracy in Cs predictions, while ADAM performs better in capturing localized variations of τ*, especially in high-resolution datasets. These findings highlight the trade-offs between precision and efficiency in machine learning-enhanced SGS models for LES applications.