dibyajyotic commited on
Commit
5f7da7e
·
verified ·
1 Parent(s): 510dce4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +198 -0
README.md CHANGED
@@ -1,3 +1,201 @@
1
  ---
2
  license: openmdw-1.1
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: openmdw-1.1
3
  ---
4
+ # Model Overview
5
+
6
+ ## Description:
7
+
8
+ corrdiff-era5-hrrr is a high-resolution generative downscaling (super-resolution) model. It predicts 99 state variables at km scale over the Continental United States (CONUS) from coarse global ERA5 reanalysis, with dense vertical resolution in the atmospheric boundary layer. It shares the state variables, conditioning variables, HRRR grid, and Diffusion Transformer (DiT) backbone with neighborhood attention (NATTEN) of StormCast-CONUS. corrdiff-era5-hrrr super-resolves ERA5 onto the HRRR grid at the same timestep, and trains with rectified flow matching. Repeated sampling from the same ERA5 input produces an ensemble of km-scale states consistent with the coarse input.
9
+
10
+ For training recipes see [NVIDIA PhysicsNeMo](https://github.com/NVIDIA/physicsnemo), for inference see [NVIDIA Earth2Studio](https://github.com/NVIDIA/earth2studio).
11
+
12
+ This model is ready for commercial or non-commercial use.
13
+
14
+ ### License/Terms of Use:
15
+
16
+ **Governing Terms**: Use of this model is governed by the [Linux Foundation OpenMDW License Agreement, version 1.1](https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1).
17
+
18
+ ### Deployment Geography:
19
+
20
+ Global
21
+
22
+ ### Use Case:
23
+
24
+ Km-scale downscaling of global reanalysis and ensemble generation over CONUS, for renewable-energy resource assessment, climate and catastrophe-risk research, hydrology, and atmospheric science
25
+
26
+ ### Release Date:
27
+
28
+ Hugging Face 09/28/2026 via [https://huggingface.co/nvidia/corrdiff-era5-hrrr](https://huggingface.co/nvidia/corrdiff-era5-hrrr)
29
+
30
+ ## Reference(s):
31
+
32
+ - CorrDiff: [Residual Diffusion Modeling for Km-scale Atmospheric Downscaling](https://arxiv.org/abs/2309.15214) <br>
33
+ - StormCast v1: [Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling](https://arxiv.org/abs/2408.10958) <br>
34
+
35
+ ## Model Architecture:
36
+
37
+ **Architecture Type:** corrdiff-era5-hrrr uses a Diffusion Transformer (DiT) architecture with neighborhood attention (NATTEN), 678M parameters. <br>
38
+ **Network Architecture:** Diffusion Transformer (DiT) with Neighborhood Attention (NATTEN) <br>
39
+ **This model was developed based on:** Not Applicable. The model was trained from scratch on paired ERA5/HRRR data; no pre-trained base model was used. <br>
40
+ **Number of model parameters:** 6.8 × 10<sup>8</sup> <br>
41
+
42
+ ## Input:
43
+
44
+ **Input Type(s):**
45
+
46
+ - Tensor (125 Surface and Model level variables - 99 state variables and 26 conditioning variables.)
47
+ - DateTime (NumPy Array) <br>
48
+
49
+ **Input Format(s):** PyTorch Tensor / NumPy array <br>
50
+ **Input Parameters:**
51
+
52
+ - Four Dimensional (4D) (batch, variable, latitude, longitude) <br>
53
+ - Input DateTime (1D) <br>
54
+
55
+ **Other Properties Related to Input:**
56
+
57
+ - Input latitude/longitude grid: HRRR Lambert Conformal Conic projection, 3 km resolution, 1024×1792 grid
58
+ - Input state weather variables: `u10m`, `v10m`, `t2m`, `mslp`, `u_hl1`, `u_hl2`, `u_hl3`, `u_hl4`, `u_hl5`, `u_hl6`, `u_hl7`, `u_hl8`, `u_hl9`, `u_hl10`, `u_hl11`, `u_hl13`, `u_hl15`, `u_hl20`, `u_hl25`, `u_hl30`, `v_hl1`, `v_hl2`, `v_hl3`, `v_hl4`, `v_hl5`, `v_hl6`, `v_hl7`, `v_hl8`, `v_hl9`, `v_hl10`, `v_hl11`, `v_hl13`, `v_hl15`, `v_hl20`, `v_hl25`, `v_hl30`, `t_hl1`, `t_hl2`, `t_hl3`, `t_hl4`, `t_hl5`, `t_hl6`, `t_hl7`, `t_hl8`, `t_hl9`, `t_hl10`, `t_hl11`, `t_hl13`, `t_hl15`, `t_hl20`, `t_hl25`, `t_hl30`, `q_hl1`, `q_hl2`, `q_hl3`, `q_hl4`, `q_hl5`, `q_hl6`, `q_hl7`, `q_hl8`, `q_hl9`, `q_hl10`, `q_hl11`, `q_hl13`, `q_hl15`, `q_hl20`, `q_hl25`, `q_hl30`, `z_hl1`, `z_hl2`, `z_hl3`, `z_hl4`, `z_hl5`, `z_hl6`, `z_hl7`, `z_hl8`, `z_hl9`, `z_hl10`, `z_hl11`, `z_hl13`, `z_hl15`, `z_hl20`, `z_hl25`, `z_hl30`, `p_hl1`, `p_hl2`, `p_hl3`, `p_hl4`, `p_hl5`, `p_hl6`, `p_hl7`, `p_hl8`, `p_hl9`, `p_hl10`, `p_hl11`, `p_hl13`, `p_hl15`, `p_hl20`, `refc` <br>
59
+ - Conditioning weather variables: `u10m`, `v10m`, `t2m`, `tcwv`, `mslp`, `sp`, `u1000`, `u850`, `u500`, `u250`, `v1000`, `v850`, `v500`, `v250`, `z1000`, `z850`, `z500`, `z250`, `t1000`, `t850`, `t500`, `t250`, `q1000`, `q850`, `q500`, `q250`
60
+ - Cosine of solar zenith angle (computed)
61
+ - Constant inputs: mean and standard deviation of surface elevation in pixel; land/water mask
62
+
63
+ For variable naming information, review the `HRRR` Lexicon at [Earth2Studio](https://github.com/NVIDIA/earth2studio/blob/main/earth2studio/lexicon/hrrr.py), but `u`, `v`, `t`, `z` and `p` refer to winds, temperature, geopotential, and pressure (respectively). Variables marked with `_hl` refer to natural/hybrid model levels.
64
+
65
+ ## Output:
66
+
67
+ **Output Type(s):** Tensor (99 Surface and Model level variables) <br>
68
+ **Output Format:** PyTorch Tensors <br>
69
+ **Output Parameters:** Four Dimensional (4D) (batch, variable, latitude, longitude) <br>
70
+ **Other Properties Related to Output:**
71
+
72
+ - Output latitude/longitude grid: HRRR Lambert Conformal Conic projection, 3 km resolution, 1024×1792 grid
73
+ - Output state weather variables: `u10m`, `v10m`, `t2m`, `mslp`, `u_hl1`, `u_hl2`, `u_hl3`, `u_hl4`, `u_hl5`, `u_hl6`, `u_hl7`, `u_hl8`, `u_hl9`, `u_hl10`, `u_hl11`, `u_hl13`, `u_hl15`, `u_hl20`, `u_hl25`, `u_hl30`, `v_hl1`, `v_hl2`, `v_hl3`, `v_hl4`, `v_hl5`, `v_hl6`, `v_hl7`, `v_hl8`, `v_hl9`, `v_hl10`, `v_hl11`, `v_hl13`, `v_hl15`, `v_hl20`, `v_hl25`, `v_hl30`, `t_hl1`, `t_hl2`, `t_hl3`, `t_hl4`, `t_hl5`, `t_hl6`, `t_hl7`, `t_hl8`, `t_hl9`, `t_hl10`, `t_hl11`, `t_hl13`, `t_hl15`, `t_hl20`, `t_hl25`, `t_hl30`, `q_hl1`, `q_hl2`, `q_hl3`, `q_hl4`, `q_hl5`, `q_hl6`, `q_hl7`, `q_hl8`, `q_hl9`, `q_hl10`, `q_hl11`, `q_hl13`, `q_hl15`, `q_hl20`, `q_hl25`, `q_hl30`, `z_hl1`, `z_hl2`, `z_hl3`, `z_hl4`, `z_hl5`, `z_hl6`, `z_hl7`, `z_hl8`, `z_hl9`, `z_hl10`, `z_hl11`, `z_hl13`, `z_hl15`, `z_hl20`, `z_hl25`, `z_hl30`, `p_hl1`, `p_hl2`, `p_hl3`, `p_hl4`, `p_hl5`, `p_hl6`, `p_hl7`, `p_hl8`, `p_hl9`, `p_hl10`, `p_hl11`, `p_hl13`, `p_hl15`, `p_hl20`, `refc` <br>
74
+
75
+ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
76
+
77
+ ## Software Integration
78
+
79
+ **Runtime Engine(s):** Not Applicable <br>
80
+ **Supported Hardware Microarchitecture Compatibility:** <br>
81
+ * NVIDIA Ampere <br>
82
+ * NVIDIA Blackwell <br>
83
+ * NVIDIA Hopper <br>
84
+
85
+ **Supported Operating System(s):**
86
+ * Linux <br>
87
+
88
+ The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
89
+
90
+ ## Model Version(s):
91
+
92
+ **Model Version:** v0 <br>
93
+
94
+ # Training, Testing, and Evaluation Datasets:
95
+
96
+ ## Training Dataset:
97
+
98
+ **Link:** [ERA5](https://cds.climate.copernicus.eu/) <br>
99
+
100
+ *Data Collection Method by dataset:* <br>
101
+ * Automatic/Sensors <br>
102
+
103
+ *Labeling Method by dataset:* <br>
104
+ * Automatic/Sensors <br>
105
+
106
+ *Data Modality:*
107
+ * Gridded geophysical time series
108
+
109
+ *Data Size:*
110
+ * 150 GB subset used for model training
111
+
112
+ **Properties:**
113
+ ERA5 data for the period 2018-07-12 – 2025-09-30. ERA5 provides hourly estimates of various
114
+ atmospheric, land, and oceanic climate variables. The data covers the Earth on a 30km
115
+ grid and resolves the atmosphere at 137 levels. <br>
116
+
117
+ **Link:** [HRRR](https://rapidrefresh.noaa.gov/hrrr/) <br>
118
+
119
+ *Data Collection Method by dataset:* <br>
120
+ * Automatic/Sensors <br>
121
+
122
+ *Labeling Method by dataset:* <br>
123
+ * Automatic/Sensors <br>
124
+
125
+ *Data Modality:*
126
+ * Gridded geophysical time series
127
+
128
+ *Data Size:*
129
+ * 15 TB subset used for model training
130
+
131
+ **Properties:**
132
+ HRRR data for the date range of 2018-07-12 to 2025-09-30. The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3 km grids with 3 km radar assimilation. <br>
133
+
134
+ **Link:** [ASTER GDEM/ASTWBD](https://asterweb.jpl.nasa.gov/gdem.asp) <br>
135
+
136
+ *Data Collection Method by dataset:* <br>
137
+ * Automatic/Sensors <br>
138
+
139
+ *Labeling Method by dataset:* <br>
140
+ * Automatic/Sensors <br>
141
+
142
+ *Data Modality:*
143
+ * Gridded geophysical static data
144
+
145
+ *Data Size:*
146
+ * 120 MB upsampled subset used for model training
147
+
148
+ **Properties:**
149
+ Global digital elevation model (DEM) and water body map. Native resolution ~30 m/pixel, upsampled to HRRR grid. <br>
150
+
151
+ ## Testing/Evaluation Dataset:
152
+
153
+ **Link:** [ERA5](https://cds.climate.copernicus.eu/) <br>
154
+
155
+ *Data Collection Method by dataset:* <br>
156
+ * Automatic/Sensors <br>
157
+
158
+ *Labeling Method by dataset:* <br>
159
+ * Automatic/Sensors <br>
160
+
161
+ **Properties:**
162
+ ERA5 data for the date range of 2025-10-01 - 2025-12-31. ERA5 provides hourly estimates of various
163
+ atmospheric, land, and oceanic climate variables. The data covers the Earth on a 30km
164
+ grid and resolves the atmosphere at 137 levels. <br>
165
+
166
+ **Link:** [HRRR](https://rapidrefresh.noaa.gov/hrrr/) <br>
167
+
168
+ *Data Collection Method by dataset:* <br>
169
+ * Automatic/Sensors <br>
170
+
171
+ *Labeling Method by dataset:* <br>
172
+ * Automatic/Sensors <br>
173
+
174
+ **Properties:**
175
+ HRRR data for the date range of 2025-10-01 - 2025-12-31. The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3 km grids with 3 km radar assimilation. <br>
176
+
177
+ **Link:** [ASTER GDEM/ASTWBD](https://asterweb.jpl.nasa.gov/gdem.asp) <br>
178
+
179
+ *Data Collection Method by dataset:* <br>
180
+ * Automatic/Sensors <br>
181
+
182
+ *Labeling Method by dataset:* <br>
183
+ * Automatic/Sensors <br>
184
+
185
+ **Properties:**
186
+ Global digital elevation model (DEM) and water body map. Native resolution ~30 m/pixel, upsampled to HRRR grid. <br>
187
+
188
+ ## Inference:
189
+
190
+ **Acceleration Engine:** [PyTorch](https://pytorch.org/) / [Earth2Studio](https://github.com/NVIDIA/earth2studio) <br>
191
+ **Test Hardware:**
192
+ * NVIDIA Hopper (H100 80 GB) <br>
193
+
194
+
195
+ ## Ethical Considerations:
196
+
197
+ NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
198
+
199
+ For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](https://huggingface.co/nvidia/corrdiff-era5-hrrr/blob/main/explainability.md), [Bias](https://huggingface.co/nvidia/corrdiff-era5-hrrr/blob/main/bias.md), [Safety & Security](https://huggingface.co/nvidia/corrdiff-era5-hrrr/blob/main/safety.md), and [Privacy](https://huggingface.co/nvidia/corrdiff-era5-hrrr/blob/main/privacy.md) Subcards.
200
+
201
+ Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns through the [NVIDIA security vulnerability submission form](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).