海洋与大气领域大模型研究进展-海洋预报

海洋与大气领域大模型研究进展

单位：1. 中国海洋大学海洋与大气学院, 山东青岛 266100;
2. 中国海洋大学三亚海洋研究院, 海南三亚 572025;
3. 国家海洋环境预报中心, 北京 100081;
4. 国家海洋环境预报中心自然资源部海洋灾害预报技术重点实验室, 北京 100081;
5. 青岛埃克曼科技有限公司, 山东

关键词：深度学习数值预报 Transformer 大模型

分类号：P732.6

出版年·卷·期（页码）：2025·42·第二期（109-122）

摘要：

详细探讨了近年来全球尺度上主要的海洋和大气深度学习预报模型，尤其是基于Transformer架构的自回归模型。按照模型的提出时间，重点介绍了FourCastNet、Pangu-Weather、ClimaX、GraphCast、FengWu、FuXi和AI-GOMS模型的基本架构、训练数据以及预报性能。通过比较这些模型，揭示其发展的主要趋势，即从简单的Transformer结构到引入AFNO、SwinTransformer、GNN等不同架构，模型在结构上不断创新；同时，模型在输入变量选择、训练策略和计算效率方面也不断优化，提升了预报的准确性和实用性。展望未来，随着图形处理器（GPU）技术的进步和跨学科合作的加强，预计模型将在提高预报技巧、延长预报时效和揭示新物理机制等方面取得进一步突破，同时模型也将进一步提高计算效率和降低资源需求。

This review comprehensively examines the major deep learning models applied to global-scale ocean and atmospheric forecasting in recent years, with a particular emphasis on autoregressive models based on the Transformer architecture. We focus on analyzing models such as FourCastNet, Pangu-Weather, ClimaX, GraphCast, FengWu, FuXi, and AI-GOMS, chronologically detailing their fundamental architectures, training data, and forecasting performance. Through a comparative analysis of these models, this paper elucidates the primary trends in model development: from simple Transformer structures to the incorporation of diverse architectures such as AFNO, Swin Transformer, and GNN, demonstrating continuous structural innovation. Concurrently, models have been optimized in terms of input variable selection, training strategies, and computational efficiency to enhance forecast accuracy and practical applicability. Looking ahead, with advancements in GPU (Graphics Processing Unit) technology and strengthened interdisciplinary collaboration, we anticipate further breakthroughs in improving forecasting skills, extending prediction lead times, and uncovering new physical mechanisms, while simultaneously focusing on enhancing computational efficiency and reducing resource requirements.

参考文献：

CHIZAS C N, MICHAELIDES S, PATTICHIS C S, et al. Artificial neural networks in forecasting minimum temperature (weather)[C]//1991 Second International Conference on Artificial Neural Networks. Bournemouth:IET, 1991:112-114.
[2] COMRIE A C. Comparing neural networks and regression models for ozone forecasting[J]. Journal of the Air & Waste Management Association, 1997, 47(6):653-663.
[3] HALL T, BROOKS H E, DOSWELL III C A. Precipitation forecasting using a neural network[J]. Weather and Forecasting, 1999, 14(3):338-345.
[4] DEO M C, NAIDU C S. Real time wave forecasting using neural networks[J]. Ocean Engineering, 1998, 26(3):191-203.
[5] GAO S, HUANG J, LI Y R, et al. A forecasting model for wave heights based on a long short-term memory neural network[J]. Acta Oceanologica Sinica, 2021, 40(1):62-69.
[6] ZHOU S Y, BETHEL B J, SUN W J, et al. Improving significant wave height forecasts using a joint empirical mode decompositionlong short-term memory network[J]. Journal of Marine Science and Engineering, 2021, 9(7):744.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach:Curran Associates Inc., 2017:6000-6010.
[8] PATHAK J, SUBRAMANIAN S, HARRINGTON P, et al. Fourcastnet:A global data-driven high-resolution weather model using adaptive fourier neural operators[J]. arXiv:2202.11214.
[9] RASP S, THUEREY N. Data - driven medium - range weather prediction with a resnet pretrained on climate simulations:A new model for weatherbench[J]. Journal of Advances in Modeling Earth Systems, 2021, 13(2):e2020MS002405.
[10] WEYN J A, DURRAN D R, CARUANA R. Improving data - driven global weather prediction using deep convolutional neural networks on a cubed sphere[J]. Journal of Advances in Modeling Earth Systems, 2020, 12(9):e2020MS002109.
[11] LI Z Y, KOVACHKI N B, AZIZZADENESHELI K, et al. Fourier neural operator for parametric partial differential equations[C]//9th International Conference on Learning Representations. ICLR, 2021.
[12] GUIBAS J, MARDANI M, LI Z Y, et al. Adaptive fourier neural operators:Efficient token mixers for transformers[J]. arXiv:2111.13587.
[13] BI K F, XIE L X, ZHANG H H, et al. Pangu-weather:A 3d highresolution model for fast and accurate global weather forecast[J]. arXiv:2211.02556.
[14] BI K F, XIE L X, ZHANG H H, et al. Accurate medium-range global weather forecasting with 3D neural networks[J]. Nature, 2023, 619(7970):533-538.
[15] LIU Z, LIN Y T, CAO Y, et al. Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2021:9992-10002.
[16] NGUYEN T, BRANDSTETTER J, KAPOOR A, et al. ClimaX:A foundation model for weather and climate[C]//Proceedings of the 40th International Conference on Machine Learning. Honolulu:ICML, 2023:25904-25938.
[17] RASP S, DUEBEN P D, SCHER S, et al. WeatherBench:a benchmark data set for data-driven weather forecasting[J]. Journal of Advances in Modeling Earth Systems, 2020, 12(11):e2020MS002203.
[18] LAM R, SANCHEZ-GONZALEZ A, WILLSON M, et al. GraphCast:Learning skillful medium-range global weather forecasting[J]. arXiv:2212.12794.
[19] LAM R, SANCHEZ-GONZALEZ A, WILLSON M, et al. Learning skillful medium-range global weather forecasting[J]. Science, 2023, 382(6677):1416-1421.
[20] CHEN K, HAN T, GONG J C, et al. FengWu:Pushing the skillful global medium-range weather forecast beyond 10 days lead[J]. arXiv:2304.02948.
[21] KRASHINSKY R, GIROUX O, JONES S, et al. Nvidia ampere architecture in-depth[EB/OL]. NVIDIA blog, 2020. https://devblogs.nvidia.com/nvidia-ampere-architecture-in-depth.
[22] CHEN L, ZHONG X H, ZHANG F, et al. FuXi:A cascade machine learning forecasting system for 15-day global weather forecast[J]. npj Climate and Atmospheric Science, 2023, 6(1):190.
[23] LIU Z, HU H, LIN Y T, et al. Swin transformer v2:Scaling up capacity and resolution[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans:IEEE, 2022:11999-12009.
[24] RONNEBERGER O, FISCHER P, BROX T. U-net:Convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference. Munich:Springer, 2015:234-241.
[25] XIONG W, XIANG Y F, WU H, et al. AI-GOMS:Large AI-driven global ocean modeling system[J]. arXiv:2308.03152.
[26] CYBENKO G. Approximation by superpositions of a sigmoidal function[J]. Mathematics of Control, Signals and Systems, 1989, 2(4):303-314.
[27] HORNIK K. Approximation capabilities of multilayer feedforward networks[J]. Neural Networks, 1991, 4(2):251-257.
[28] RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physicsinformed neural networks:A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational Physics, 2019, 378:686-707.
[29] DAW A, KARPATNE A, WATKINS W D, et al. Physics-guided neural networks (PGNN):An application in lake temperature modeling[M]//KARPATNE A, KANNAN R, KUMAR V. Knowledge Guided Machine Learning. New York:Chapman and Hall/CRC, 2022:353-372.
[30] KOCHKOV D, YUVAL J, LANGMORE I, et al. Neural general circulation models for weather and climate[J]. Nature, 2024, 632(8027):1060-1066.
[31] WANG X, WANG R Z, HU N Z, et al. XiHe:A data-driven model for global ocean eddy-resolving forecasting[J]. arXiv:2402.02995.
[32] LEVIN L A, BETT B J, GATES A R, et al. Global observing needs in the deep ocean[J]. Frontiers in Marine Science, 2019, 6:241.
[33] VANDAL T, KODRA E, GANGULY S, et al. DeepSD:Generating high resolution climate change projections through single image super-resolution[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax:Association for Computing Machinery, 2017:1663-1672.
[34] PAN B X, HSU K, AGHAKOUCHAK A, et al. Improving precipitation estimation using convolutional neural network[J]. Water Resources Research, 2019, 55(3):2301-2321.
[35] LIANG Y X, KE S Y, ZHANG J B, et al. GeoMAN:Multi-level attention networks for geo-sensory time series prediction[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm:AAAI Press, 2018:3428- 3434.

服务与反馈：

【文章下载】【发表评论】【查看评论】【加入收藏】