Numerical models require adjustments ("model tuning") to allow for uncertainty due to incomplete knowledge of physical processes, algorithmic approximations, unresolved processes and their integration/interaction with the overall model. To that end, forecasts are commonly summarized into scalar quantities (e.g., mean across forecast domain), in terms of which the effect of the model parameters is examined. However, recent advent of spatial verification methods calls for a focus on spatial features of the forecasts, and how these features are affected by model parameters. This research aims to develop a methodology for addressing this problem.
Such a framework is necessary and timely because its ingredients have already emerged. Although the topic of spatial verification is new and rapidly changing, many of its elements have crystallized into fixtures. For instance, it is now clear that verification of precipitation must account for "objects" within the forecast field, and that the assessment of forecast errors must address errors in magnitude, spatial placement, and orientation of such objects. This study examines features/objects which can be objectively identified with statistical methods (e.g., clustering) developed by the PIs for spatial verification.
Techniques for assessing the effect of model parameters on specific forecast features are also in rapid development. This study focuses on a class of methods for sensitivity analysis, called variance-based (or global). The PIs have already applied this method to a number of examples, including the Lorenz's 63 model, and some non-spatial and semi-spatial features of precipitation forecasts from an operational mesoscale weather forecast model. Here, a variance-based sensitivity analysis method will be adapted to spatial forecasts; the method is ideally suited to problems where there are nonlinear relationships and interactions between multiple model parameters and multiple forecast features. The method will be tested on Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) and Weather Research and Forecasting model (WRF).
Intellectual Merit: The intellectual merit of the work is in extending the notion of model tuning to align it with the basic tenets of spatial verification. The framework will improve forecasts by allowing a user to tune the model parameters to have desirable spatial features. The PIs have contributed to the development of the spatial techniques as well as to the sensitivity analysis methods suitable for this problem.
Broader Impacts: The broader impacts of the research are multi-pronged. The student involved in this project will become an expert in some of the most sophisticated topics in numerical prediction models: spatial verification, sensitivity analysis, and model tuning. This highly interdisciplinary framework itself has general applicability in that it can be employed to improve predictions from any numerical model involving inexact parameterizations of physical processes or inexact knowledge of physical constants, including weather, climate, ocean, etc. Given that spatial verification is emerging as the verification method of choice for future forecasts, the method will have wide-spread use. The method will be disseminated in the form of journal articles, and computer code to be included in a verification R package produced by National Center for Atmospheric Research (NCAR).
