Assimilating Conventional Observations
Conventional observations are assimilated from PREPBUFR files. NCEP ADP Global Upper Air and Surface Weather Observations (PREPBUFR format) are composed of a global set of surface and upper air reports operationally collected by the National Centers for Environmental Prediction (NCEP). These include land surface, marine surface, radiosonde, pibal and aircraft reports from the Global Telecommunications System (GTS), profiler and US radar derived winds, SSM/I oceanic winds and TCW retrievals, and satellite wind data from the National Environmental Satellite Data and Information Service (NESDIS). The reports can include pressure, geopotential height, temperature, dew point temperature, wind direction and speed [@cisl_rda_ds337.0].
While the PREPBUFR includes wind derived from satellite observations, GSI ignores this observations and uses the ones provided by the specific bufr file gdas.t00z.satwnd.tm00.bufr_d.
PREPBUFR files usually contains observations from a 6 to 12 h window and can be modify using FORTRAN routines provided with the GSI code (see util/bufr_tools in the GSI source code folder). You can also create your own bufr file or add new observation to an existing bufr file (see Working with bufr files).
Controlling which observations are assimilated
The assimilation of conventional observations is controlled with the convinfo file. Let’s check the global_convinfo.txt file we get as an example in the fix folder:
! otype = observation type (a7, t, uv, q, etc.)
! type = prepbufr observation type (if available)
! sub = prepbufr subtype (not yet available)
! iuse = flag if to use/not use / monitor data
! = 1 use data
! = 0 do not use data
! = -1 monitor data
! twindow = time window (+/- hours)
! numgrp = cross validation parameter - number of groups
! ngroup = cross validation parameter - group to remove from data use
! nmiter = cross validation parameter - external iteration to introduce removed data
! gross = gross error parameter - gross error
! ermax = gross error parameter - max error
! ermin = gross error parameter - min error
! var_b = variational quality control parameter - b parameter
! var_pg ithin rmesh npred = variational quality control parameter - pg parameter
! pmot: the optione to keep thinned datai as monitored, 0: not to keep, other values: to keep
! ptime: time interval for thinning, 0, no temporal thinning, other values define time interval (less than 6)
!otype type sub iuse twindow numgrp ngroup nmiter gross ermax ermin var_b var_pg ithin rmesh pmesh npred pmot ptime
tcp 112 0 1 3.0 0 0 0 75.0 5.0 1.0 75.0 0.000000 0 0. 0. 0 0. 0.
ps 120 0 1 3.0 0 0 0 4.0 3.0 1.0 4.0 0.000300 0 0. 0. 0 0. 0.
ps 132 0 -1 3.0 0 0 0 4.0 3.0 1.0 4.0 0.000300 0 0. 0. 0 0. 0.
ps 180 0 1 3.0 0 0 0 4.0 3.0 1.0 4.0 0.000300 0 0. 0. 0 0. 0.
ps 180 01 1 3.0 0 0 0 4.0 3.0 1.0 4.0 0.000300 0 0. 0. 0 0. 0.The head of the file explains the content of each column but there are a few more things to add:
- type: this is defined by the bufr tables, particular Table 2. It is worth checking this table as includes information about which observations are assimilated in GFS, errors associated to specific instruments and other details.
- twindow: while the assimilation window is defined in the gsi namalist, it is possible to control an assimilation window for specific observations. This is useful if, for example the assimilation window is 3 h and you want to assimilate temperature in a 1h window.
In general you only change the iuse column to assimilate or not a type of observation and maybe just maybe the gross, ermax, and ermin parameters if you want to modify the quality control of the observations.
Observation errors and quality control
For regional assimilation GSI uses an error table located in the errtable file that you’ll find in the ./fix folder under the name prepobs_errtable.global (yes, it is confusing that the table for regional errors in in a file called global). Here is a small example of the content of the file for observations from surface stations.
181 OBSERVATION TYPE
0.11000E+04 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.10500E+04 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.10000E+04 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.95000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.90000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.85000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.80000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.75000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10
0.70000E+03 0.15000E+01 0.20000E+01 0.10000E+10 0.16000E+01 0.10000E+10The meaning of each column is described in the following table:
| Column # | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| Content | Pressure | T | RH | UV | Ps | Pw |
| Unit | hPa | degreeC | percent/10 | m/s | mb | kg/m2 (or mm) |
So column 1 define the pressure level associated to the errors for each variable in columns 2 to 6. In the errtable a 0.10000E+10 is a NA.
GSI will perform a quality control for each observation. In general terms this involves a gross check and specific controls depending on the type of observation.
For the gross check, GSI first calculates a ratio:
\[ ratio = (obs - bk)/max(ermin, min(ermax, obserror)) \] The main error parameters are controlled by the convinfo file. The obserror is the observation error defined in the prepbufr file for each observation plus information in the prepobs_errtable.
If \(ration > gross\) the observation is rejected.
Other piece of information used during the quality control is the quality control flag that is included in the prepbufr file a part if it quality control process performed by NCEP. The possible values for conventional observations are:
| qc flag | meaning |
|---|---|
| 0-2 | –> Obs is assimilated |
| 3 | Suspicious obs –> gross check is more strict |
| 4-15 | –> Obs is rejected (for some cases 9 o 15 means that the obs is monitored |
You can find more details about the quality control flags in Table 7.
GSI can also perform a thinning for conventional observations. You can activate that option for each type of observation changing ithin = 1 in the convinfo file. There are other important columns, rmesh, pmesh, in the convinfo file to configure conventional data thinning:
ithin: 0 = no thinning; 1 = thinning with grid mesh decided byrmeshandpmeshrmesh: horizontal thinning grid size in kmpmesh: vertical thinning grid size in mb; if 0, then use background vertical grid
For each observation GSI will check different things and change the observation error accordingly. The final observation error is recorded in the diag file and then used during the assimilation.