Development of a skill assessment tool for the Korea operational oceanographic system
doi: 10.1007/s13131-013-0354-9
Development of a skill assessment tool for the Korea operational oceanographic system
-
摘要: A standard skill assessment (SA) tool was developed and implemented to evaluate the performance of operational forecast models in the Korea operational oceanographic system. The SA tool provided a robust way to assess model skill in the system by comparing predictions and observations, and involved the computation of multiple skillmetrics including correlation and error skills. User-and system-based acceptance criteria of skill metrics were applied to determine whether predictions were acceptable for the system. To achieve this, the tool produced a time series comparison plot, a skill score table, and an advanced summarized diagram to effectively demonstrate the multiple skill scores. Moreover, the SA was conducted to evaluate both atmospheric and hydrodynamic forecast variables. For the atmospheric variables, acceptable error criteriawere preferable to acceptable correlation criteria over short timescales, since themean square error overwhelmed the observation variance. Conversely, for the hydrodynamic variables, acceptable root mean square percentage error (e.g., perms) criteria were preferable to acceptable error (e.g.,erms) criteria owing to the spatially variable tidal intensity around the Korean Peninsula. Furthermore, the SA indicated that predetermined acceptance error criteria were appropriate to satisfy a target central frequency (fc) for which errors fell within the specified limits (i.e., the fc equals 70%).Abstract: A standard skill assessment (SA) tool was developed and implemented to evaluate the performance of operational forecast models in the Korea operational oceanographic system. The SA tool provided a robust way to assess model skill in the system by comparing predictions and observations, and involved the computation of multiple skillmetrics including correlation and error skills. User-and system-based acceptance criteria of skill metrics were applied to determine whether predictions were acceptable for the system. To achieve this, the tool produced a time series comparison plot, a skill score table, and an advanced summarized diagram to effectively demonstrate the multiple skill scores. Moreover, the SA was conducted to evaluate both atmospheric and hydrodynamic forecast variables. For the atmospheric variables, acceptable error criteriawere preferable to acceptable correlation criteria over short timescales, since themean square error overwhelmed the observation variance. Conversely, for the hydrodynamic variables, acceptable root mean square percentage error (e.g., perms) criteria were preferable to acceptable error (e.g.,erms) criteria owing to the spatially variable tidal intensity around the Korean Peninsula. Furthermore, the SA indicated that predetermined acceptance error criteria were appropriate to satisfy a target central frequency (fc) for which errors fell within the specified limits (i.e., the fc equals 70%).
-
Anthes R A, Kuo Y-H, Hsie E-Y, et al. 1989. Estimation of skill and uncertainty in regional numerical models. Q J Roy Meteor Soc, 115: 763-806 Cancino L, Neves R. 1999. Hydrodynamic and sediment suspension modeling in estuarine systems: Part I. Description of the numerical models. JMar Syst, 22: 105-116 Chen C, Liu H, Beardsley R C. 2003. An unstructured grid, finitevolume, three-dimensional, primitive equations ocean model: Application to coastal ocean and estuaries. J Atmos Ocean Tech, 20: 159-186 HaidvogelD B, ArangoH G,Hedstrom K, et al. 2000. Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates. Dynam Atmos Oceans, 32: 239-281 Hess K W, Gross T F, Schmalz R A, et al. 2003. NOS standards for evaluating operational nowcast and forecast hydrodynamic model systems. NOAA Tech Rep NOS CS, 17: 1-48 Jolliff J K, Kindle J C, Shulman I, et al. 2009. Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment. J Mar Syst, 76: 64-82 Kim S D, Kim B, Baek S-H. 2011. Data archiving system for the operational ocean forecasting. The Second China-Korea Joint Workshop on Marine Environmental Forecasting for the Yellow Sea and East China Sea, Vol. 1: 85-86 Lee J-C, Kwon J-I, Park K-W, et al. 2008. Calculations of storm surges, Typhoon Maemi. J Korean Soc Coast Ocean Eng, 20: 93-100 Legates D R, McCabe Jr G J. 1999. Evaluating the use of “goodness-offit” measures in hydrologic and hydroclimatic model validation. Water Resour Res, 35(1): 233-241 Loague K, Green R E. 1991. Statistical and graphical methods for evaluating solute transport models: overview and application. J ContamHydrol, 7: 51-73 Matsumoto K, Takanezawa T, Ooe M. 2000. Ocean tide models developed by assimilating TOPEX/POSEIDON altimeter data into hydrodynamical model: A global model and a regional model around Japan. J Oceanogr, 56: 567-581 Murphy AH, Epstein E S. 1989. Skill scores and correlation coefficients in model verification. MonWeather Rev, 117: 572-581 Park K S, Lee J C, Jun K C, et al. 2009. Development of an operational storm surge prediction system for the Korean Coast. Ocean and Polar Res, 31(4): 369-377 Stow C A, Jolliff J, McGillicuddy Jr D J, et al. 2009. Skill assessment for coupled biological/physical models of marine systems. J Mar Syst, 76: 4-15 Stow C A, Roessler C, BorsukME, et al. 2003. A comparison of estuarine water quality models for TMDL development in the Neuse River Estuary. JWater Res PL-ASCE, 129: 307-314 Taylor K E. 2001. Summarizing multiple aspects of model performance in a single diagram. J Geophys Res, 106(D7): 7183-7192 Willmott C J, Ackleson S G, Davis R E, et al. 1985. Statistics for the evaluation and comparison of models. J Geophys Res, 90(C5): 8995-9005 Willmott C J. 1981. On the validation of models. Phys Geogr, 2: 184-194 Zhang A, Hess K W, Aikman III F. 2010. User-based skill assessment techniques for operational hydrodynamic forecast systems. J Oper Oceanogr, 3: 11-24 Zhang A, Hess K W, Wei E, et al. 2006. Implementation of model skill assessment software for water level and current in tidal regions. NOAA Tech Rep NOS CS, 24: 1-61
点击查看大图
计量
- 文章访问数: 1276
- HTML全文浏览量: 90
- PDF下载量: 1626
- 被引次数: 0