10. Parameterization-specific Output¶
10.1. Overview¶
When used with UFS and the SCM, the CCPP offers the capability of outputting tendencies of temperature, zonal wind, meridional wind, ozone, and specific humidity produced by the parameterizations of selected suites. This capability is useful for understanding the behavior of the individual parameterizations in terms of magnitude and spatial distribution of tendencies, which can help model developers debug, refine, and tune their schemes.
The CCPP also enables outputting two-dimensional (2D) or three-dimensional (3D) arbitrary diagnostics from the parameterizations. This capability is targeted to model developers who may benefit from analyzing intermediate quantities computed in one or more parameterizations. One example of a desirable diagnostic is tendencies from sub-processes within a parameterization, such as the tendencies from condensation, evaporation, sublimation, etc. from a microphysics parameterization. The output is done using CCPP-provided 2D- and 3D arrays, and the developer can fill positions 1, 2, .., N of the array. Important aspects of the implementation are that memory is only allocated for the necessary positions of the array and that all diagnostics are output on physics model levels. An extension to enable output on radiation levels may be considered in future implementations.
These capabilities have been tested and are expected to work with the following suites:
SCM: GFS_v16, GFS_v17_p8, RAP, RRFS_v1beta, WoFS, HRRR
ufs-weather-model (regional): GFS_v16, RRFS_v1beta, WoFS, HRRR
10.2. Tendencies¶
This section describes the tendencies available, how to set the model to prepare them and how to output them. It also contains a list of frequently-asked questions in Section 10.2.6.
10.2.1. Available Tendencies¶
The model can produce tendencies for temperature, wind, and all non-chemical tracers (see Table 10.1) for several different schemes. Not all schemes produce all tendencies. For example, the orographic and convective gravity wave drag (GWD) schemes produce tendencies of temperature and wind, but not of tracers. Similarly, only the planetary boundary layer (PBL), deep and shallow convection, and microphysics schemes produce specific humidity tendencies. Some PBL and convection schemes will have tendencies for tracers, and others won’t.
In addition to the tendencies from specific schemes, the output includes tendencies from all photochemical processes, all physics processes, and all non-physics processes (last three rows of Table 10.2). Examples of non-physical processes are dynamical core processes such as advection and nudging toward climatological fields.
In the supported suites, there are two types of schemes that produce ozone tendencies: PBL and ozone photochemistry. The total tendency produced by the ozone photochemistry scheme (NRL 2015 scheme) is subdivided by subprocesses: production and loss (combined as a single subprocess), quantity of ozone present in the column above a grid cell, influences from temperature, and influences from mixing ratio. For more information about the NRL 2015 ozone photochemistry scheme, consult the CCPP Scientific Documentation.
There are numerous tendencies in CCPP, and you need to know which ones exist for your configuration to enable them. The model will output a list of available tendencies for your configuration if you run with diagnostic tendencies enabled. To avoid overusing memory, you should enable just one tendency, which is available for all suites, the non-physics (ie. dynamics) tendency of temperature. Details of how to do this, and how to use the information, is below.
10.2.2. Enabling Tendencies¶
For performance reasons, the preparation of tendencies for output is off by default in the UFS and can be turned on via a set of namelist options. Since the SCM is not operational and has a relatively tiny memory footprint, these tendencies are turned on by default in the SCM.
There are three namelist variables associated with this capability: ldiag3d
, qdiag3d
, and
dtend_select
. These are set in the &gfs_physics_nml
portion of the namelist file input.nml
.
ldiag3d
enables tendencies for state variables (horizontal wind and temperature)qdiag3d
enables tendencies for tracers;ldiag3d
must also be enableddtend_select
enables only a subset of the tendencies turned on byldiag3d
andqdiag3d
If dtend_select
is not specified, the default is to select all tendencies enabled by the settings of
ldiag3d
and qdiag3d
.
Note that there is a fourth namelist variable, lssav
, associated with the output of
parameterization-specific information. The value of lssav
is overwritten to true in the code, so the
value used in the namelist is irrelevant.
While the tendencies output by the SCM are instantaneous, the tendencies output by the UFS are averaged
over the number of hours specified by the user in variable fhzero
in the &gfs_physics_nml
portion of the
namelist file input.nml
. Variable fhzero
must be an integer (it cannot be zero).
This example namelist selects all tendencies from microphysics processes, and all tendencies of
temperature. The naming convention for dtend_select
is explained in the next section.
&gfs_physics_nml
ldiag3d = .true. ! enable basic diagnostics
qdiag3d = .true. ! also enable tracer diagnostics
dtend_select = 'dtend*mp', 'dtend_temp_*' ! Asterisks (*) and question marks (?) have the same meaning as shell globs
! The default for dtend_select is '*' which selects everything
! ... other namelist parameters ...
/
10.2.3. Tendency Names¶
Tendency variables follow the naming pattern below, which is used to enable calculation (input.nml
) and output
of the variable:
dtend_variable_process
The dtend_
string stands for “diagnostic tendency” and is used to avoid variable name clashes. Replace
variable
with the short name of the tracer or state variable (see Table 10.1). Replace process
with the short name of the process that is changing the
variable (see Table 10.2). For example, microphysics (mp
) temperature
(temp
) tendency is dtend_temp_mp
.
The next section will tell you how to determine which tendency variables are available for your model.
Variable |
Short Name |
Associated Namelist Variables |
dtidx Index |
Tendency Units |
---|---|---|---|---|
Temperature |
|
|
|
K s-1 |
X Wind |
|
|
|
m s-2 |
Y Wind |
|
|
|
m s-2 |
Water Vapor Specific Humidity |
|
|
|
kg kg-1 s-1 |
Ozone Concentration |
|
|
|
kg kg-1 s-1 |
Cloud Condensate or Liquid Water |
|
|
|
kg kg-1 s-1 |
Ice Water |
|
|
|
kg kg-1 s-1 |
Rain Water |
|
|
|
kg kg-1 s-1 |
Snow Water |
|
|
|
kg kg-1 s-1 |
Graupel |
|
|
|
kg kg-1 s-1 |
Cloud Amount |
|
|
|
kg kg-1 s-1 |
Liquid Number Concentration |
|
|
|
kg-1 s-1 |
Ice Number Concentration |
|
|
|
kg-1 s-1 |
Rain Number Concentration |
|
|
|
kg-1 s-1 |
Snow Number Concentration |
|
|
|
kg-1 s-1 |
Graupel Number Concentration |
|
|
|
kg-1 s-1 |
Turbulent Kinetic Energy |
|
|
|
J s-1 |
Mass Weighted Rime Factor |
|
|
|
kg kg-1 s-1 |
Number Concentration Of Water-Friendly Aerosols |
|
|
|
kg-1 s-1 |
Number Concentration Of Ice-Friendly Aerosols |
|
|
|
kg-1 s-1 |
Oxygen Ion Concentration |
|
|
|
kg kg-1 s-1 |
Oxygen Concentration |
|
|
|
kg kg-1 s-1 |
Process |
diag_table Module Name |
Short Name |
dtidx Index |
---|---|---|---|
Planetary Boundary Layer |
|
|
|
Deep Convection |
|
|
|
Shallow Convection |
|
|
|
Microphysics |
|
|
|
Convective Transport |
|
|
|
Long Wave Radiation |
|
|
|
Short Wave Radiation |
|
|
|
Orographic Gravity Wave Drag |
|
|
|
Rayleigh Damping |
|
|
|
Convective Gravity Wave Drag |
|
|
|
Production and Loss (Photochemical) |
|
|
|
Ozone Mixing Ratio (Photochemical) |
|
|
|
Temperature-Induced (Photochemical) |
|
|
|
Overhead Ozone Column (Photochemical) |
|
|
|
Sum of Photochemical Processes |
|
|
|
Sum of Physics Processes (Including Photochemical) |
|
|
|
Sum of Non-Physics Processes |
|
|
|
10.2.4. Selecting Tendencies¶
With the many suites and many combinations of schemes, it is hard to say which variable/process combinations are available for your particular configuration. To find a list, enable diagnostics, but disable all tracer/process combinations except one:
&gfs_physics_nml
ldiag3d = .true. ! enable basic diagnostics
qdiag3d = .true. ! also enable tracer diagnostics
dtend_select = 'dtend_temp_nophys' ! All configurations have non-physics temperature tendencies
! ... other namelist parameters ...
/
After recompiling and running the model, you will see lines like this in the model’s standard output stream:
dtend selected: gfs_phys dtend_qv_mp = water vapor specific humidity tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_liq_wat_mp = cloud condensate (or liquid water) tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_rainwat_mp = rain water tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_ice_wat_mp = ice water tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_snowwat_mp = snow water tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_graupel_mp = graupel tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_cld_amt_mp = cloud amount integer tendency due to microphysics (kg kg-1 s-1)
dtend selected: gfs_phys dtend_temp_phys = temperature tendency due to physics (K s-1)
dtend selected: gfs_dyn dtend_temp_nophys = temperature tendency due to non-physics processes (K s-1)
There are three critical pieces of information in each line. Taking the third last line as an example,
dtend_cld_amt_mp
– this is the name of the variable indtend_select
; for the UFS, it is also the name of the variable in thediag_table
gfs_phys
– thediag_table
module name (specific to the UFS, can be ignored for other models)“cloud amount integer tendency due to microphysics” – meaning of the variable
Note that the dtend_temp_nophys
differs from the others in that it is in the gfs_dyn
module
instead of gfs_phys
because it sums non-physics processes. This is only relevant for the UFS.
Now that you know what variables are available, you can choose which to enable:
&gfs_physics_nml
ldiag3d = .true. ! enable basic diagnostics
qdiag3d = .true. ! also enable tracer diagnostics
dtend_select = 'dtend*mp', 'dtend_temp_*' ! Asterisks (*) and question marks (?) have the same meaning as shell globs
! The default for dtend_select is '*' which selects everything
! ... other namelist parameters ...
/
Note that any combined tendencies, such as the total temperature tendency from physics (dtend_temp_phys
),
will only include other tendencies that were calculated. Hence, if you only calculate PBL and microphysics
tendencies then your “total temperature tendency” will actually just be the total of PBL and microphysics.
The third step is to enable output of variables, which will be discussed in the next section.
10.2.5. Outputting Tendencies¶
10.2.5.1. UFS¶
After enabling tendency calculation (using ldiag3d
, qdiag3d
, and diag_select
), you must also
enable output of those tendencies using the diag_table
. Enter the new lines with the variables you want
output. Continuing our example from before, this will enable output of some microphysics tracer tendencies,
and the total tendencies of temperature:
"gfs_phys", "dtend_qv_mp", "dtend_qv_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_liq_wat_mp", "dtend_liq_wat_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_rainwat_mp", "dtend_rainwat_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_ice_wat_mp", "dtend_ice_wat_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_snowwat_mp", "dtend_snowwat_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_graupel_mp", "dtend_graupel_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_cld_amt_mp", "dtend_cld_amt_mp", "fv3_history", "all", .false., "none", 2
"gfs_phys", "dtend_temp_phys", "dtend_temp_phys", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "dtend_temp_nophys", "dtend_temp_nophys", "fv3_history", "all", .false., "none", 2
Note that all tendencies, except non-physics tendencies, are in the gfs_phys
diagnostic module. The
non-physics tendencies are in the gfs_dyn
module. This is reflected in the Table 10.2.
Note that some host models, such as the UFS, have a limit of how many fields can be output in a run.
When outputting all tendencies, this limit may have to be increased. In the UFS, this limit is determined
by variable max_output_fields
in namelist section &diag_manager_nml
in file input.nml
.
Further documentation of the diag_table
file can be found in the UFS Weather Model User’s Guide.
When the model completes, the fv3_history file will contain these new variables.
10.2.5.2. SCM¶
The default behavior of the SCM is to output instantaneous values of all tendency variables, and
dtend_select
is not recognized. Tendencies are computed in file scm_output.F90
in the
subroutines output_init and output_append. If the values of ldiag3d
or qdiag3d
are set to false, the
variables are still written to output but are given missing values.
10.2.6. FAQ¶
10.2.6.1. What is the meaning of error message max_output_fields
was exceeded?¶
If the limit to the number of output fields is exceeded, the job may fail with the following message:
FATAL from PE 24: diag_util_mod::init_output_field: max_output_fields = 300 exceeded. Increase via diag_manager_nml
In this case, increase max_output_fields
in input.nml
:
&diag_manager_nml
prepend_date = .F.
max_output_fields = 600
10.2.7. Why did I run out of memory when outputting tendencies?¶
Trying to output all tendencies may use more memory than is available on your system. Use dtend_select
and choose your output
variables carefully!
10.2.8. Why did I get a runtime logic error when outputting tendencies?¶
Setting ldiag3d=F
and qdiag3d=T
will result in an error message:
Logic error in GFS_typedefs.F90: qdiag3d requires ldiag3d
If you want to output tracer tendencies, you must set both ldiag3d
and qdiag3d
to T. Then use
diag_select
to enable only the tendencies you want. Make sure your diag_table
matches your choice of tendencies specified through diag_select
.
10.2.9. Why are my tendencies zero, even though the model says they are supported for my configuration?¶
For total physics or total photochemistry tendencies, see the next question.
The tendencies will be zero if they are never calculated. Check that you enabled the tendencies with
appropriate settings of ldiag3d
, qdiag3d
, and diag_select
.
Another possibility is that the tendencies in question really are zero. The list of “available” tendencies is set at the model level, where the exact details of schemes and suites are not known. This can lead to some tendencies erroneously being listed as available. For example, some PBL schemes have ozone tendencies and some don’t, so some may have zero ozone tendencies. Also, some schemes don’t have tendencies of state variables or tracers. Instead, they modify different variables, which other schemes use to affect the state variables and tracers. Unfortunately, not all of the 3D variables in CCPP have diagnostic tendencies.
10.2.10. Why are my total physics or total photochemistry tendencies zero?¶
There are three likely reasons:
You forgot to enable calculation of physics tendencies. Make sure
ldiag3d
andqdiag3d
are T, and make surediag_select
selects physics tendencies.The suite did not enable the
phys_tend
scheme, which calculates the total physics and total photochemistry tendencies.You did not enable calculation of the individual tendencies, such as ozone. The
phys_tend
sums those to make the total tendencies.
10.3. Output of Auxiliary Arrays from CCPP¶
The output of diagnostics from one or more parameterizations involves changes to the
namelist and code changes in the parameterization(s) (to load the desirable information
onto the CCPP-provided arrays and to add them to the subroutine arguments) and in the
parameterization metadata descriptor file(s) (to provide metadata on the new subroutine
arguments). In the UFS, the namelist is used to control the temporal averaging period.
These code changes are intended to be used by scientists during the development process
and are not intended to be incorporated into the authoritative code. Therefore, developers
must remove any code related to these additional diagnostics before submitting a pull
request to the ccpp-physics
repository.
The auxiliary diagnostics from CCPP are output in arrays:
aux2d - auxiliary 2D array for outputting diagnostics
aux3d - auxiliary 3D array for outputting diagnostics
and dimensioned by:
naux2d - number of 2D auxiliary arrays to output for diagnostics
naux3d - number of 3D auxiliary arrays to output diagnostics
At runtime, these arrays will be written to the output files. Note that auxiliary arrays can be output from more than one parameterization in a given run.
The UFS and SCM already contain code to declare and initialize the arrays:
dimensions are declared and initialized in
GFS_typedefs.F90
metadata for these arrays and dimensions are defined in
GFS_typedefs.meta
arrays are populated in
GFS_diagnostics.F90
(UFS) orscm_output.F90
(SCM)
The remainder of this section describes changes the developer needs to make in the physics code and in the host model control files to enable the capability. An example (Section 10.3.2) and FAQ (Section 10.3.2.1) are also provided.
10.3.1. Enabling the auxiliary arrays capability¶
10.3.1.1. Physics-side changes¶
In order to output auxiliary arrays, developers need to change at least the following two files within the physics (see also example in Section 10.3.2):
- A CCPP entrypoint scheme (Fortran source code)
Add array(s) and its/their dimension(s) to the list of subroutine arguments
Declare array(s) with appropriate intent and dimension(s). Note that array(s) do not need to be allocated by the developer. This is done automatically in
GFS_typedefs.F90
.Populate array(s) with desirable diagnostic for output
- Associated CCPP metadata files for modified scheme(s)
Add entries for the array(s) and its/their dimension(s) and provide metadata
10.3.1.2. Host-side changes¶
10.3.1.2.1. UFS¶
For the UFS, developers have to change the following two files on the host side (also see example provided in Section 10.3.2)
- Namelist file
input.nml
Specify how many 2D and 3D arrays will be output using variables
naux2d
andnaux3d
in section&gfs_physics_nml
, respectively. The maximum allowed number of arrays to output is 20 2D and 20 3D arrays.Specify whether the output should be for instantaneous or time-averaged quantities using variables
aux2d_time_avg
andaux_3d_time_avg
. These arrays are dimensionednaux2d
andnaux3d
, respectively, and, if not specified in the namelist, take the default value F.Specify the period of averaging for the arrays using variable
fhzero
(in hours).
- Namelist file
- File
diag_table
Enable output of the arrays at runtime.
2D and 3D arrays are written to the output files.
- File
10.3.1.2.2. SCM¶
Typically, in a 3D model, 2D arrays represent variables with two horizontal dimensions, e.g. x and y, whereas 3D arrays represent variables with all three spatial dimensions, e.g. x, y, and z. For the SCM, these arrays are implicitly 1D and 2D, respectively, where the “y” dimension is 1 and the “x” dimension represents the number of independent columns (typically also 1). For continuity with the UFS Atmosphere, the naming convention 2D and 3D are retained, however. With this understanding, the namelist files can be modified as in the UFS:
- Namelist file
input.nml
Specify how many 2D and 3D arrays will be output using variables
naux2d
andnaux3d
in section&gfs_physics_nml
, respectively. The maximum allowed number of arrays to output is 20 2D and 20 3D arrays.Unlike the UFS, only instantaneous values are output. Time-averaging can be done through post-processing the output. Therefore, the values of
aux2d_time_avg
andaux_3d_time_avg
should not be changed from their default false values. As such, the namelist variablefhzero
has no effect in the SCM.
- Namelist file
10.3.2. Recompiling and Examples¶
The developer must recompile the code after making the source code changes to the CCPP scheme(s) and associated metadata files. Changes in the namelist and diag table can be made after compilation. At compile and runtime, the developer must pick suites that use the scheme from which output is desired.
An example for how to output auxiliary arrays is provided in the rest of this section. The lines that
start with “+” represent lines that were added by the developer to output the diagnostic arrays. In
this example, the developer modified the Grell-Freitas (GF) cumulus scheme to output two 2D arrays
and one 3D array. The 2D arrays are aux_2d (:,1)
and aux_2d(:,2)
; the 3D array is aux_3d(:,:,1)
.
The 2D array aux2d(:,1)
will be output with an averaging in time in the UFS, while the aux2d(:,2)
and aux3d
arrays will not be averaged.
In this example, the arrays are populated with bogus information just to demonstrate the capability. In reality, a developer would populate the array with the actual quantity for which output is desirable.
diff --git a/physics/cu_gf_driver.F90 b/physics/cu_gf_driver.F90
index 927b452..aed7348 100644
--- a/physics/cu_gf_driver.F90
+++ b/physics/cu_gf_driver.F90
@@ -76,7 +76,8 @@ contains
flag_for_scnv_generic_tend,flag_for_dcnv_generic_tend, &
du3dt_SCNV,dv3dt_SCNV,dt3dt_SCNV,dq3dt_SCNV, &
du3dt_DCNV,dv3dt_DCNV,dt3dt_DCNV,dq3dt_DCNV, &
- ldiag3d,qdiag3d,qci_conv,errmsg,errflg)
+ ldiag3d,qdiag3d,qci_conv,errmsg,errflg, &
+ naux2d,naux3d,aux2d,aux3d)
!-------------------------------------------------------------
implicit none
integer, parameter :: maxiens=1
@@ -137,6 +138,11 @@ contains
integer, intent(in ) :: imfshalcnv
character(len=*), intent(out) :: errmsg
integer, intent(out) :: errflg
+
+ integer, intent(in) :: naux2d,naux3d
+ real(kind_phys), intent(inout) :: aux2d(:,:)
+ real(kind_phys), intent(inout) :: aux3d(:,:,:)
+
! define locally for now.
integer, dimension(im),intent(inout) :: cactiv
integer, dimension(im) :: k22_shallow,kbcon_shallow,ktop_shallow
@@ -199,6 +205,11 @@ contains
! initialize ccpp error handling variables
errmsg = ''
errflg = 0
+
+ aux2d(:,1) = aux2d(:,1) + 1
+ aux2d(:,2) = aux2d(:,2) + 2
+ aux3d(:,:,1) = aux3d(:,:,1) + 3
+
!
! Scale specific humidity to dry mixing ratio
!
The cu_gf_driver.meta
file was modified accordingly:
diff --git a/physics/cu_gf_driver.meta b/physics/cu_gf_driver.meta
index 99e6ca6..a738721 100644
--- a/physics/cu_gf_driver.meta
+++ b/physics/cu_gf_driver.meta
@@ -476,3 +476,29 @@
type = integer
intent = out
+[naux2d]
+ standard_name = number_of_2d_auxiliary_arrays
+ long_name = number of 2d auxiliary arrays to output (for debugging)
+ units = count
+ dimensions = ()
+ type = integer
+[naux3d]
+ standard_name = number_of_3d_auxiliary_arrays
+ long_name = number of 3d auxiliary arrays to output (for debugging)
+ units = count
+ dimensions = ()
+ type = integer
+[aux2d]
+ standard_name = auxiliary_2d_arrays
+ long_name = auxiliary 2d arrays to output (for debugging)
+ units = none
+ dimensions = (horizontal_loop_extent,number_of_3d_auxiliary_arrays)
+ type = real
+ kind = kind_phys
+[aux3d]
+ standard_name = auxiliary_3d_arrays
+ long_name = auxiliary 3d arrays to output (for debugging)
+ units = none
+ dimensions = (horizontal_loop_extent,vertical_layer_dimension,number_of_3d_auxiliary_arrays)
+ type = real
+ kind = kind_phys
The following lines were added to the &gfs_physics_nml
section of the namelist file input.nml
:
naux2d = 2
naux3d = 1
aux2d_time_avg = .true., .false.
Recall that for the SCM, aux2d_time_avg
should not be set to true in the namelist.
Lastly, the following lines were added to the diag_table
for UFS:
# Auxiliary output
"gfs_phys", "aux2d_01", "aux2d_01", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "aux2d_02", "aux2d_02", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "aux3d_01", "aux3d_01", "fv3_history", "all", .false., "none",
10.3.2.1. FAQ¶
10.3.2.1.1. How do I enable the output of diagnostic arrays from multiple parameterizations in a single run?¶
Suppose you want to output two 2D arrays from schemeA and two 2D arrays from schemeB. You should
set the namelist to naux2d=4
and naux3d=0
. In the code for schemeA, you should populate
aux2d(:,1)
and aux2d(:,2)
, while in the code for scheme B you should populate aux2d(:,3)
and aux2d(:,4)
.