A deep learning model for ocean surface latent heat flux based on Transformer and data assimilation
-
Abstract: Efficient and accurate prediction of ocean surface latent heat fluxes is essential for understanding and modeling climate dynamics. Conventional estimation methods have low resolution and lack accuracy. The Transformer model, with its self-attention mechanism, effectively captures long-range dependencies, leading to a degradation of accuracy over time. Due to the non-linearity and uncertainty of physical processes, the Transformer model encounters the problem of error accumulation, leading to a degradation of accuracy over time. To solve this problem, we combine the data assimilation technique with the Transformer model and continuously modify the model state to make it closer to the actual observations. In this paper, we propose a deep learning model called TransNetDA, which integrates Transformer, Convolutional Neural Network and data assimilation methods. By combining data-driven and data assimilation methods for spatiotemporal prediction, TransNetDA effectively extracts multi-scale spatial features and significantly improves prediction accuracy. The experimental results indicate that the TransNetDA method surpasses traditional techniques in terms of RMSE and R2 metrics, showcasing its superior performance in predicting latent heat fluxes at the ocean surface.
-
Key words:
- climate dynamics /
- deep learning /
- data assimilation /
- Transformer /
- ensemble kalman filter /
- ocean surface latent heat flux
1) Conflict of interest: -
Figure 8. Comparison of 01:30, 07:30 and 13:30 real data (left), hourly TransNetDA prediction (center), and the differences between them (right) on January 1, 2021. DA frequency is opted as 6 hours for this experiment.
Figure 9. Scatterplot of TransNetDA model predictions vs. true data on January 1, 2021.
Figure 10. Comparison of 01:30, 07:30 and 13:30 real data (left), hourly TransNet prediction (center), and the differences between them (right) on January 1, 2021.
Figure 11. Evaluate the prediction accuracy of the TransNetDA on January 1, 2021 at different DA frequencies (6, 12 and 24 hours).
Table 1. Training setting of the model
Module Setting Size Datasets Input dimension (100, 80, 1) Output dimension (100, 80, 1) Training/Validation/Test Split 2000~2019/
2020/2021Encoder Number of Layer 4 Number of Transformer Blocks [2,3,4,4] Embedding Dimensions [32,64,128,256] Norm layer LayerNorm Decoder Number of Layer 4 Number of Transformer Blocks [2,3,4,4] Embedding Dimensions [32,64,128,256] 
下载: 导出CSV
Table 2. The
$ {\mathrm{R}}^{2} $ and RMSE values of four models on January 1, 2021 at 01:30, 04:30, and 07:30, respectively.Model 01:30 04:30 07:30 $ {\mathrm{R}}^{2} $ RMSE $ {\mathrm{R}}^{2} $ RMSE $ {\mathrm{R}}^{2} $ RMSE Persistence 0.899 18.716 0.956 9.993 0.962 8.739 Climatology 0.471 42.851 0.406 41.935 0.339 43.109 U-Net 0.973 6.945 0.907 11.875 0.859 14.215 LSTM 0.929 10.354 0.863 13.801 0.786 17.633 ConvLSTM 0.955 8.283 0.894 12.315 0.835 15.504 TransNetDA 0.997 2.385 0.970 7.227 0.985 4.785
下载: 导出CSV
-
Bonan G B, Doney S C. 2018. Climate, ecosystems, and planetary futures: The challenge to predict life in earth system models. Science, 359(6375): eaam8328, doi: 10.1126/science.aam8328 Bouttier F, Courtier P. 2002. Data assimilation concepts and methods March 1999. Meteorological Training Course Lecture Series, ECMWF,718: 59 Brajard J, Carrassi A, Bocquet M, et al. 2020. Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model. Journal of Computational Science, 44: 101171, doi: 10.1016/j.jocs.2020.101171 Carrassi A, Bocquet M, Bertino L, et al. 2018. Data assimilation in the geosciences: An overview of methods, issues, and perspectives. WIREs Climate Change, 9(5): e535, doi: 10.1002/wcc.535 Chen Qiaochuan, Cai Candong, Chen Yaoran, et al. 2024. TemproNet: A transformer-based deep learning model for seawater temperature prediction. Ocean Engineering, 293: 116651, doi: 10.1016/j.oceaneng.2023.116651 Chen Xiaowei, Yao Yunjun, Li Yufu, et al. 2020. ANN-based estimation of low-latitude monthly ocean latent heat flux by ensemble satellite and reanalysis products. Sensors, 20(17): 4773, doi: 10.3390/s20174773 Chen Xueen, Zhan Peng, Chen Jinrui, et al. 2011. Numerical study of current fields near the Changjiang Estuary and impact of Quick-EnKF assimilation. Acta Oceanologica Sinica, 30(5): 33–44, doi: 10.1007/s13131-011-0145-0 Cheng Meiling, Fang Fangxin, Navon I M, et al. 2023a. Ensemble Kalman filter for GAN-ConvLSTM based long lead-time forecasting. Journal of Computational Science, 69: 102024, doi: 10.1016/j.jocs.2023.102024 Cheng Sibo, Quilodrán-Casas C, Ouala S, et al. 2023b. Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA Journal of Automatica Sinica, 10(6): 1361–1387, doi: 10.1109/JAS.2023.123537 Chicco D, Warrens M J, Jurman G. 2021. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Computer Science, 7: e623, doi: 10.7717/peerj-cs.623 Durand M, Margulis S A. 2006. Feasibility test of multifrequency radiometric data assimilation to estimate snow water equivalent. Journal of Hydrometeorology, 7(3): 443–457, doi: 10.1175/JHM502.1 Erturk A, Inman D J. 2008. Issues in mathematical modeling of piezoelectric energy harvesters. Smart Materials and Structures, 17(6): 065016, doi: 10.1088/0964-1726/17/6/065016 Evensen G. 2003. The Ensemble Kalman Filter: Theoretical formulation and practical implementation. Ocean Dynamics, 53(4): 343–367, doi: 10.1007/s10236-003-0036-9 Geer A J, Lonitz K, Weston P, et al. 2018. All-sky satellite data assimilation at operational weather forecasting centres. Quarterly Journal of the Royal Meteorological Society, 144(713): 1191–1217, doi: 10.1002/qj.3202 Gharamti M E, Tjiputra J, Bethke I, et al. 2017. Ensemble data assimilation for ocean biogeochemical state and parameter estimation at different sites. Ocean Modelling, 112: 65–89, doi: 10.1016/j.ocemod.2017.02.006 Gong Daoyi, Wang Shaowu. 1998. Response of global subtropical highs to the equatorial eastern Pacific SST anomaly. Acta Oceanologica Sinica (in Chinese), 20(5): 44–54 Guo Xiaozheng, Yao Yunjun, Tang Qingxin, et al. 2024. Multimodel ensemble estimation of Landsat-like global terrestrial latent heat flux using a generalized deep CNN-LSTM integration algorithm. Agricultural and Forest Meteorology, 349: 109962, doi: 10.1016/j.agrformet.2024.109962 Han Kai, Xiao An, Wu Enhua, et al. 2021. Transformer in transformer. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. Online: Curran Associates Inc. , 15908–15919 Hochreiter S, Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8): 1735–1780, doi: 10.1162/neco.1997.9.8.1735 Jurisic B, Uglesic I, Xemard A, et al. 2018. High frequency transformer model derived from limited information about the transformer geometry. International Journal of Electrical Power & Energy Systems, 94: 300–310 Kessler R C, van Loo H M, Wardenaar K J, et al. 2016. Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Molecular Psychiatry, 21(10): 1366–1371, doi: 10.1038/mp.2015.198 Krasnopolsky V M, Chevallier F. 2003. Some neural network applications in environmental sciences. Part II: Advancing computational efficiency of environmental numerical models. Neural Networks, 16(3–4): 335–348, doi: 10.1016/S0893-6080(03)00026-1 Large W G, Yeager S G. 2012. On the observed trends and changes in global sea surface temperature and air–sea heat fluxes (1984–2006). Journal of Climate, 25(18): 6123–6135, doi: 10.1175/JCLI-D-11-00148.1 Le Gallo M, Khaddam-Aljameh R, Stanisavljevic M, et al. 2023. A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference. Nature Electronics, 6(9): 680–693, doi: 10.1038/s41928-023-01010-1 Li Xin, Liu Feng, Ma Chunfeng, et al. 2024. Land data assimilation: Harmonizing theory and data in land surface process studies. Reviews of Geophysics, 62(1): e2022RG000801, doi: 10.1029/2022RG000801 Liang Miaoling, Yuan Xing, Wang Wenyan. 2023. Improving the forecasts of surface latent heat fluxes and surface air temperature in the grapes global forecast system. Atmosphere, 14(8): 1241, doi: 10.3390/atmos14081241 Lindgren F, Bolin D, Rue H. 2022. The SPDE approach for Gaussian and non-Gaussian fields: 10 years and still running. Spatial Statistics, 50: 100599, doi: 10.1016/j.spasta.2022.100599 Liu Shan, Jing Xueyi, Chen Xingrong, et al. 2023. An assessment of the subduction rate in the CMIP6 historical experiment. Acta Oceanologica Sinica, 42(1): 44–60, doi: 10.1007/s13131-022-2108-z Liu Hao, Nie Xunwei, Shi Junqiang, et al. 2024. Marine heatwaves and cold spells in the Brazil Overshoot show distinct sea surface temperature patterns depending on the forcing. Communications Earth & Environment, 5(1): 102 Liu Danian, Shi Ping, Shu Yeqiang, et al. 2016. Assimilating temperature and salinity profiles using Ensemble Kalman Filter with an adaptive observation error and T-S constraint. Acta Oceanologica Sinica, 35(1): 30–37, doi: 10.1007/s13131-016-0793-1 Liu Y, Weerts A H, Clark M, et al. 2012. Advancing data assimilation in operational hydrologic forecasting: progresses, challenges, and emerging opportunities. Hydrology and Earth System Sciences, 16(10): 3863–3887, doi: 10.5194/hess-16-3863-2012 Lorenc A C. 2003. The potential of the ensemble Kalman filter for NWP-A comparison with 4D-var. Quarterly Journal of the Royal Meteorological Society, 129(595): 3183–3203, doi: 10.1256/qj.02.132 Madani N, Kimball J S, Parazoo N C, et al. 2020. Below-surface water mediates the response of African forests to reduced rainfall. Environmental Research Letters, 15(3): 034063, doi: 10.1088/1748-9326/ab724a Malik P, Vachharajani B H, Panicker D V. 2024. Logistic curve modelling of sea surface temperature and latent heat flux variability in the Tropical Indian Ocean. Dynamics of Atmospheres and Oceans, 107: 101467, doi: 10.1016/j.dynatmoce.2024.101467 Maulik R, Rao V, Wang Jiali, et al. 2022. Efficient high-dimensional variational data assimilation with machine-learned reduced-order models. Geoscientific Model Development, 15(8): 3433–3445, doi: 10.5194/gmd-15-3433-2022 Muhuri P S, Chatterjee P, Yuan Xiaohong, et al. 2020. Using a long short-term memory recurrent neural network (LSTM-RNN) to classify network attacks. Information, 11(5): 243, doi: 10.3390/info11050243 Niu Zhaoyang, Zhong Guoqiang, Yu Hui. 2021. A review on the attention mechanism of deep learning. Neurocomputing, 452: 48–62, doi: 10.1016/j.neucom.2021.03.091 Oke P R, Brassington G B, Griffin D A, et al. 2008. The Bluelink ocean data assimilation system (BODAS). Ocean Modelling, 21(1–2): 46–70, doi: 10.1016/j.ocemod.2007.11.002 Olsson F, Moore T N, Carey C C, et al. 2024. A multi-model ensemble of baseline and process-based models improves the predictive skill of near-term lake forecasts. Water Resources Research, 60(3): e2023WR035901, doi: 10.1029/2023WR035901 Pan Ming, Wood E F, Wójcik R, et al. 2008. Estimation of regional terrestrial water cycle using multi-sensor remote sensing observations and data assimilation. Remote Sensing of Environment, 112(4): 1282–1294, doi: 10.1016/j.rse.2007.02.039 Pettorelli N, Schulte to Bühne H, Tulloch A, et al. 2018. Satellite remote sensing of ecosystem functions: opportunities, challenges and way forward. Remote Sensing in Ecology and Conservation, 4(2): 71–93, doi: 10.1002/rse2.59 Rafieeinasab A, Seo D J, Lee H, et al. 2014. Comparative evaluation of maximum likelihood ensemble filter and ensemble Kalman filter for real-time assimilation of streamflow data into operational hydrologic models. Journal of Hydrology, 519: 2663–2675, doi: 10.1016/j.jhydrol.2014.06.052 Reichle R H. 2008. Data assimilation methods in the Earth sciences. Advances in Water Resources, 31(11): 1411–1418, doi: 10.1016/j.advwatres.2008.01.001 Reichstein M, Camps-Valls G, Stevens B, et al. 2019. Deep learning and process understanding for data-driven earth system science. Nature, 566(7743): 195–204, doi: 10.1038/s41586-019-0912-1 Ruiz J J, Pulido M, Miyoshi T. 2013. Estimating model parameters with ensemble-based data assimilation: A review. Journal of the Meteorological Society of Japan, 91(2): 79–99 Salahuddin Z, Woodruff H C, Chatterjee A, et al. 2022. Transparency of deep neural networks for medical image analysis: A review of interpretability methods. Computers in Biology and Medicine, 140: 105111, doi: 10.1016/j.compbiomed.2021.105111 Shan Hongming, Zhang Yi, Yang Qingsong, et al. 2018. 3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network. IEEE Transactions on Medical Imaging, 37(6): 1522–1534, doi: 10.1109/TMI.2018.2832217 Stergiou A, Poppe R. 2023. AdaPool: Exponential adaptive pooling for information-retaining downsampling. IEEE Transactions on Image Processing, 32: 251–266, doi: 10.1109/TIP.2022.3227503 Tang Shengquan, Chen Xueen, Zeng Zhi, et al. 2022. Numerical investigation of the South China Sea deep circulation. Acta Oceanologica Sinica, 41(5): 1–11, doi: 10.1007/s13131-021-1879-y Tierney L, Mira A. 1999. Some adaptive Monte Carlo methods for Bayesian inference. Statistics in Medicine, 18(17–18): 2507–2515, doi: 10.1002/(SICI)1097-0258(19990915/30)18:17/18<2507::AID-SIM272>3.0.CO;2-J Usama M, Ahmad B, Song Enmin, et al. 2020. Attention-based sentiment analysis using convolutional and recurrent neural network. Future Generation Computer Systems, 113: 571–578, doi: 10.1016/j.future.2020.07.022 Wang Yansen, Tao Weikuo, Simpson J. 1996. The impact of ocean surface fluxes on a TOGA COARE convective system. Monthly Weather Review, 124(12): 2753–2763, doi: 10.1175/1520-0493(1996)124<2753:TIOOSF>2.0.CO;2 Ward E J, Holmes E E, Thorson J T, et al. 2014. Complexity is costly: A meta-analysis of parametric and non-parametric methods for short-term population forecasting. Oikos, 123(6): 652–661, doi: 10.1111/j.1600-0706.2014.00916.x Yang L M, Grooms I. 2021. Machine learning techniques to construct patched analog ensembles for data assimilation. Journal of Computational Physics, 443: 110532, doi: 10.1016/j.jcp.2021.110532 Yucel I, Onen A, Yilmaz K K, et al. 2015. Calibration and evaluation of a flood forecasting system: Utility of numerical weather prediction model, data assimilation and satellite-based rainfall. Journal of Hydrology, 523: 49–66, doi: 10.1016/j.jhydrol.2015.01.042 -
点击查看大图
计量
- 文章访问数: 49
- HTML全文浏览量: 19
- 被引次数: 0
