Grid Levels and Parameters

Notebook Python-AWIPS Tutorial Notebook

Objectives

Cover the relevant methods for accessing EDEX and investigating what data is available.
This example we look at the “grid” data type and investigate the Global Forcast System (GFS) model.
We will talk quite a bit about the DataAccessLayer utility, and its online documentation might be a helpful reference.

Table of Contents

1
Imports
2 Connect to
EDEX
3 Get a List of Supported Data
Types
4 Create a New Data Request and Set the
Type
5 Get Available
Locations
6 Get Available
Parameters
7 Get Available
Levels
8 Get Available
Times
9 Get the
Data!
10 See
Also
    10.1 Related
Notebooks

1 Imports

Start by importing the DataAccessLayer package from python-awips:

from awips.dataaccess import DataAccessLayer

Top

2 Connect to EDEX

Define a url for your EDEX connection, and then point python-awips at that EDEX

# Unidata's cloud EDEX instance is used in this example
edex_url = "edex-cloud.unidata.ucar.edu"
DataAccessLayer.changeEDEXHost(edex_url)

Top

3 Get a List of Supported Data Types

DataAccessLayer.getSupportedDatatypes() returns a list of supported data types offered by the EDEX server defined above. The code below shows how to populate, sort, and print out that list.

dataTypes = DataAccessLayer.getSupportedDatatypes()
dataTypes.sort()
list(dataTypes)

['acars',
 'airep',
 'binlightning',
 'bufrmosAVN',
 'bufrmosETA',
 'bufrmosGFS',
 'bufrmosHPC',
 'bufrmosLAMP',
 'bufrmosMRF',
 'bufrua',
 'climate',
 'common_obs_spatial',
 'gfe',
 'gfeEditArea',
 'grid',
 'maps',
 'modelsounding',
 'obs',
 'pirep',
 'practicewarning',
 'profiler',
 'radar',
 'radar_spatial',
 'satellite',
 'sfcobs',
 'topo',
 'warning']

Top

4 Create a New Data Request and Set the Type

Now create a new data request using DataAccessLayer.newDataRequest(), and set the data type using request.setDatatype(). Below we create a few different requests with different data types to show some differences with other methods.

For this example we are going to look at the grid data type, which is where the model data can be found, along with some other datasets (such as MRMS).

# Create a request for data type grid
grid_request = DataAccessLayer.newDataRequest()
grid_request.setDatatype("grid")

Top

5 Get Available Locations

Use the DataAccessLayer.getAvailableLocationNames(request) method to find out what locations are available for the given dataset. Typically these will be geographic locations or NWS sites, although in some instances it will be something else. Take a look at what’s outputted for the grid_request, for example.

# Grid Locations
grid_locations = DataAccessLayer.getAvailableLocationNames(grid_request)
grid_locations.sort()
list(grid_locations)

['AUTOSPE',
 'CMC',
 'ESTOFS',
 'ETSS',
 'FFG-ALR',
 'FFG-FWR',
 'FFG-KRF',
 'FFG-MSR',
 'FFG-ORN',
 'FFG-PTR',
 'FFG-RHA',
 'FFG-RSA',
 'FFG-STR',
 'FFG-TAR',
 'FFG-TIR',
 'FFG-TUA',
 'FNMOC-NCODA',
 'FNMOC-WW3',
 'GFS1p0',
 'GFS20',
 'HFR-EAST_6KM',
 'HFR-EAST_PR_6KM',
 'HFR-US_EAST_DELAWARE_1KM',
 'HFR-US_EAST_FLORIDA_2KM',
 'HFR-US_EAST_NORTH_2KM',
 'HFR-US_EAST_SOUTH_2KM',
 'HFR-US_EAST_VIRGINIA_1KM',
 'HFR-US_HAWAII_1KM',
 'HFR-US_HAWAII_2KM',
 'HFR-US_HAWAII_6KM',
 'HFR-US_WEST_500M',
 'HFR-US_WEST_CENCAL_2KM',
 'HFR-US_WEST_LOSANGELES_1KM',
 'HFR-US_WEST_LOSOSOS_1KM',
 'HFR-US_WEST_NORTH_2KM',
 'HFR-US_WEST_SANFRAN_1KM',
 'HFR-US_WEST_SOCAL_2KM',
 'HFR-US_WEST_WASHINGTON_1KM',
 'HFR-WEST_6KM',
 'HPCGuide',
 'HPCqpfNDFD',
 'HRRR',
 'LAMP2p5',
 'MRMS_0500',
 'MRMS_1000',
 'NAM12',
 'NAM40',
 'NOHRSC-SNOW',
 'RAP13',
 'RTMA',
 'RTOFS-Now-WestAtl',
 'RTOFS-Now-WestConus',
 'RTOFS-WestAtl',
 'RTOFS-WestConus',
 'SPCGuide',
 'SeaIce',
 'TPCWindProb',
 'URMA25',
 'navgem0p5']

Top

6 Get Available Parameters

We’re setting the “location” (in this case, what model we are interested in) to specify our request before we look at the available parameters.

Take a look at the available parameters for the data set by using DataAccessLayer.getAvailableParameters(request)

# Pick a model and set the location for the grid request -- we'll be using the Global Forecast System 20km (GFS20)
grid_request.setLocationNames("GFS20")
grid_params = DataAccessLayer.getAvailableParameters(grid_request)
grid_params.sort()
list(grid_params)

['36SHRMi',
 '50dbzZ',
 'AV',
 'Along',
 'AppT',
 'BLI',
 'BRN',
 'BRNEHIi',
 'BRNSHR',
 'BRNmag',
 'BRNvec',
 'BdEPT06',
 'BlkMag',
 'BlkShr',
 'CAPE',
 'CFRZR',
 'CFRZR3hr',
 'CFRZR6hr',
 'CICEP',
 'CICEP3hr',
 'CICEP6hr',
 'CIn',
 'CP',
 'CP-GFS',
 'CP3hr',
 'CP6hr',
 'CPr',
 'CPrD',
 'CRAIN',
 'CRAIN3hr',
 'CRAIN6hr',
 'CSNOW',
 'CSNOW3hr',
 'CSNOW6hr',
 'CURU',
 'CapeStk',
 'Corf',
 'CorfF',
 'CorfFM',
 'CorfM',
 'CritT1',
 'CumNrm',
 'CumShr',
 'DivF',
 'DivFn',
 'DivFs',
 'DpD',
 'DpT',
 'EHI',
 'EHI01',
 'EHIi',
 'EMSP',
 'EPT',
 'EPTA',
 'EPTC',
 'EPTGrd',
 'EPTGrdM',
 'EPTs',
 'EPVg',
 'EPVs',
 'EPVt1',
 'EPVt2',
 'ESP',
 'ESP2',
 'FVecs',
 'FnVecs',
 'FsVecs',
 'Fzra1',
 'Fzra2',
 'GH',
 'GHxSM',
 'GHxSM2',
 'GVV',
 'HI',
 'HI1',
 'HI3',
 'HI4',
 'HIdx',
 'Heli',
 'HeliC',
 'INV',
 'IPLayer',
 'Into',
 'KI',
 'L-I',
 'LIsfc2x',
 'LM5',
 'LM6',
 'MAdv',
 'MCon',
 'MCon2',
 'MLLCL',
 'MMP',
 'MSFDi',
 'MSFi',
 'MSFmi',
 'MSG',
 'MTV',
 'Mix1',
 'Mix2',
 'Mmag',
 'MnT3hr',
 'MnT6hr',
 'MpV',
 'MxT3hr',
 'MxT6hr',
 'NBE',
 'NST',
 'NST1',
 'NST2',
 'P',
 'P3hr',
 'P6hr',
 'PAdv',
 'PBE',
 'PEC',
 'PFrnt',
 'PGrd',
 'PGrd1',
 'PGrdM',
 'PIVA',
 'PTvA',
 'PTyp',
 'PW',
 'PW2',
 'PoT',
 'PoTA',
 'QPV1',
 'QPV2',
 'QPV3',
 'QPV4',
 'RH',
 'RH_001_bin',
 'RH_002_bin',
 'RM5',
 'RM6',
 'RMprop',
 'RMprop2',
 'RV',
 'Rain1',
 'Rain2',
 'Rain3',
 'Ro',
 'SH',
 'SHx',
 'SLI',
 'SNSQ',
 'SNW',
 'SNWA',
 'SRMl',
 'SRMlM',
 'SRMm',
 'SRMmM',
 'SRMr',
 'SRMrM',
 'SSP',
 'SSi',
 'STP',
 'STP1',
 'Shear',
 'ShrMag',
 'Snow1',
 'Snow2',
 'Snow3',
 'SnowT',
 'St-Pr',
 'StrTP',
 'StrmMot',
 'SuCP',
 'T',
 'TAdv',
 'TGrd',
 'TGrdM',
 'TORi',
 'TORi2',
 'TP',
 'TP-GFS',
 'TP3hr',
 'TP6hr',
 'TQIND',
 'TShrMi',
 'TV',
 'TW',
 'T_001_bin',
 'Tdef',
 'Tdend',
 'ThGrd',
 'Thom5',
 'Thom5a',
 'Thom6',
 'TmDpD',
 'Tmax',
 'Tmin',
 'Topo',
 'TotQi',
 'Tstk',
 'TwMax',
 'TwMin',
 'Twstk',
 'TxSM',
 'VAdv',
 'VAdvAdvection',
 'VGP',
 'VSS',
 'WCD',
 'WD',
 'WEASD',
 'Wind',
 'WndChl',
 'ageoW',
 'ageoWM',
 'cCape',
 'cCin',
 'cTOT',
 'capeToLvl',
 'dCape',
 'dP',
 'dT',
 'dVAdv',
 'dZ',
 'defV',
 'del2gH',
 'df',
 'fGen',
 'fnD',
 'fsD',
 'gamma',
 'gammaE',
 'geoVort',
 'geoW',
 'geoWM',
 'loCape',
 'maxEPT',
 'minEPT',
 'mixRat',
 'msl-P',
 'muCape',
 'pV',
 'pVeq',
 'qDiv',
 'qVec',
 'qnVec',
 'qsVec',
 'shWlt',
 'snoRat',
 'snoRatCrocus',
 'snoRatEMCSREF',
 'snoRatOv2',
 'snoRatSPC',
 'snoRatSPCdeep',
 'snoRatSPCsurface',
 'staticCoriolis',
 'staticSpacing',
 'staticTopo',
 'swtIdx',
 'tTOT',
 'tWind',
 'tWindU',
 'tWindV',
 'uFX',
 'uW',
 'uWStk',
 'ulSnoRat',
 'vSmthW',
 'vTOT',
 'vW',
 'vWStk',
 'wDiv',
 'wSp',
 'wSp_001_bin',
 'wSp_002_bin',
 'wSp_003_bin',
 'wSp_004_bin',
 'zAGL']

Top

7 Get Available Levels

Setting the parameters is just an option, you do not need to filter the data if you do not wish to. Also, although we are only setting one parameter in this example, you can set multiple parameters by using an array:

params = ("param1", "param2", "param3"...)
request.setParameters(params)

Set a parameter, from the output above and take a look at what “levels” are available for the data set you’re looking at using DataAccessLayer.getAvailableLevels(request).

Warning: Not all datasets support levels. If you are trying this with another dataset and run into an exception (error), it’s most likely because levels are not supported for that data type.

# For grid data we'll use the temperature parameter ("T")
grid_request.setParameters("T")
grid_levels = DataAccessLayer.getAvailableLevels(grid_request)
for lvl in grid_levels:
    print(lvl)

 0.0SFC
 350.0MB
 610.0_40000.0FHAG
 120.0_150.0BL
 900.0MB
 0.0_610.0FHAG
 450.0MB
 575.0MB
 100.0MB
 1000.0MB
 60.0_90.0BL
 1.0PV
 950.0MB
 150.0MB
 1.5PV
 700.0MB
 825.0MB
 150.0_180.0BL
 250.0MB
 1000.0_500.0MB
 800.0MB
 4000.0FHAG
 925.0MB
 2.0PV
 0.5PV
 0.0TROP
 750.0MB
 500.0MB
 625.0MB
 400.0MB
 0.0FHAG
 2.0FHAG
 875.0MB
 0.0_1000.0FHAG
 850.0MB
 600.0MB
 725.0MB
 0.0_6000.0FHAG
 975.0MB
 550.0MB
 0.0_3000.0FHAG
 675.0MB
 200.0MB
 0.0_30.0BL
 30.0_60.0BL
 650.0MB
 525.0MB
 300.0MB
 90.0_120.0BL
 1000.0FHAG
 775.0MB
 340.0_350.0K
 290.0_300.0K
 700.0_600.0MB
 700.0_300.0MB
 320.0Ke
 800.0_750.0MB
 60.0TILT
 5.3TILT
 1000.0_900.0MB
 340.0K
 5500.0_6000.0FHAG
 255.0K
 255.0_265.0K
 3000.0_6000.0FHAG
 25.0TILT
 2000.0FHAG
 0.0_500.0FHAG
 1000.0_850.0MB
 850.0_250.0MB
 280.0_290.0Ke
 1524.0FHAG
 320.0_330.0K
 0.0TILT
 310.0_320.0Ke
 310.0Ke
 330.0K
 900.0_800.0MB
 550.0_500.0MB
 2.4TILT
 50.0TILT
 3500.0FHAG
 35.0TILT
 12.0TILT
 300.0_310.0K
 3000.0_12000.0FHAG
 0.9TILT
 320.0K
 400.0_350.0MB
 500.0FHAG
 750.0_700.0MB
 1000.0_400.0MB
 345.0K
 250.0_260.0K
 300.0Ke
 290.0Ke
 950.0_900.0MB
 4572.0FHAG
 275.0_285.0Ke
 335.0Ke
 295.0_305.0Ke
 275.0_285.0K
 600.0_550.0MB
 310.0K
 9000.0FHAG
 335.0K
 1000.0_7000.0FHAG
 700.0_500.0MB
 9144.0FHAG
 325.0_335.0K
 2000.0_8000.0FHAG
 0.0_609.6FHAG
 300.0K
 315.0_325.0K
 325.0K
 340.0Ke
 0.0_4000.0FHAG
 5000.0_5500.0FHAG
 300.0_250.0MB
 1.5TILT
 335.0_345.0K
 315.0K
 3.4TILT
 2500.0FHAG
 10000.0FHAG
 0.0_2000.0FHAG
 7000.0FHAG
 5000.0FHAG
 330.0Ke
 500.0_400.0MB
 1000.0_1500.0FHAG
 305.0K
 285.0_295.0Ke
 14.0TILT
 3000.0_3500.0FHAG
 325.0_335.0Ke
 2000.0_5000.0FHAG
 7620.0FHAG
 850.0_800.0MB
 6096.0FHAG
 6000.0_7000.0FHAG
 2000.0_7000.0FHAG
 9000.0_10000.0FHAG
 295.0Ke
 305.0Ke
 265.0_275.0K
 7000.0_8000.0FHAG
 3000.0_8000.0FHAG
 700.0_650.0MB
 1000.0_6000.0FHAG
 0.5TILT
 450.0_400.0MB
 1.8TILT
 330.0_340.0K
 800.0_700.0MB
 850.0_300.0MB
 6.0TILT
 900.0_850.0MB
 3657.6FHAG
 0.0_5000.0FHAG
 320.0_330.0Ke
 8.7TILT
 650.0_600.0MB
 600.0_400.0MB
 55.0TILT
 270.0_280.0Ke
 30.0TILT
 310.0_320.0K
 1500.0FHAG
 1000.0_950.0MB
 5500.0FHAG
 250.0_200.0MB
 500.0_1000.0FHAG
 400.0_300.0MB
 500.0_100.0MB
 1000.0_3000.0FHAG
 8000.0FHAG
 285.0Ke
 290.0K
 305.0_315.0K
 285.0_295.0K
 0.0_2500.0FHAG
 925.0_850.0MB
 275.0Ke
 1500.0_2000.0FHAG
 300.0_200.0MB
 260.0_270.0K
 2743.2FHAG
 3000.0FHAG
 315.0_325.0Ke
 600.0_500.0MB
 16.7TILT
 280.0K
 500.0_250.0MB
 40.0TILT
 3048.0FHAG
 400.0_200.0MB
 300.0_310.0Ke
 270.0_280.0K
 1000.0_700.0MB
 45.0TILT
 850.0_500.0MB
 2500.0_3000.0FHAG
 609.6FHAG
 0.0_8000.0FHAG
 295.0K
 4.3TILT
 295.0_305.0K
 330.0_340.0Ke
 270.0K
 4000.0_4500.0FHAG
 280.0_290.0K
 925.0_700.0MB
 0.0_1500.0FHAG
 260.0K
 10.0TILT
 3500.0_4000.0FHAG
 325.0Ke
 285.0K
 290.0_300.0Ke
 7.5TILT
 1828.8FHAG
 280.0Ke
 500.0_450.0MB
 305.0_315.0Ke
 250.0K
 4500.0FHAG
 1250.0FHAG
 0.0_10000.0FHAG
 4500.0_5000.0FHAG
 250.0_350.0K
 270.0Ke
 275.0K
 315.0Ke
 500.0_300.0MB
 350.0_300.0MB
 750.0FHAG
 19.5TILT
 2000.0_2500.0FHAG
 850.0_700.0MB
 350.0K
 265.0K
 6000.0FHAG
 8000.0_9000.0FHAG
 700.0_300.0LYRMB
 850.0_700.0LYRMB
 1000.0_500.0LYRMB


Note - 0.0SFC is the Surface level - HAG stands for Fixed
Height Above Ground (in meters) - NTAT stands for Nominal Top of
the ATmosphere - BL stands for Boundary Layer, where
0.0_30.0BL reads as 0-30 mb above ground level - TROP is
the Tropopause level

# We'll set the level to surface level
grid_request.setLevels("0.0SFC")

Top

8 Get Available Times

Take a look at what time options are available for the data you’re looking at using the DataAccessLayer.getAvailableTimes() method:

getAvailableTimes(request, True) will return an object of run times - formatted as YYYY-MM-DD HH:MM:SS
getAvailableTimes(request) will return an object of all times - formatted as YYYY-MM-DD HH:MM:SS (F:ff)
getForecastRun(cycle, times) will return a DataTime array for a single forecast cycle.

# Available grid times
grid_cycles = DataAccessLayer.getAvailableTimes(grid_request, True)
grid_times = DataAccessLayer.getAvailableTimes(grid_request)
## Using -1 in an array will access the last element of the array
##  (using -2 will access the second last element, and so on)
grid_fcstRun = DataAccessLayer.getForecastRun(grid_cycles[-1], grid_times)

## print out the time instances
times = []
for fcst in grid_fcstRun:
    print(fcst.getRefTime(), ':', fcst.getFcstTime(), 'seconds')

2021-06-01 12:00:00.000 : 0 seconds
2021-06-01 12:00:00.000 : 10800 seconds
2021-06-01 12:00:00.000 : 21600 seconds
2021-06-01 12:00:00.000 : 32400 seconds
2021-06-01 12:00:00.000 : 43200 seconds
2021-06-01 12:00:00.000 : 54000 seconds
2021-06-01 12:00:00.000 : 64800 seconds
2021-06-01 12:00:00.000 : 75600 seconds
2021-06-01 12:00:00.000 : 86400 seconds
2021-06-01 12:00:00.000 : 97200 seconds
2021-06-01 12:00:00.000 : 108000 seconds
2021-06-01 12:00:00.000 : 118800 seconds
2021-06-01 12:00:00.000 : 129600 seconds
2021-06-01 12:00:00.000 : 140400 seconds
2021-06-01 12:00:00.000 : 151200 seconds
2021-06-01 12:00:00.000 : 162000 seconds
2021-06-01 12:00:00.000 : 172800 seconds
2021-06-01 12:00:00.000 : 183600 seconds
2021-06-01 12:00:00.000 : 194400 seconds
2021-06-01 12:00:00.000 : 205200 seconds
2021-06-01 12:00:00.000 : 216000 seconds
2021-06-01 12:00:00.000 : 226800 seconds
2021-06-01 12:00:00.000 : 237600 seconds
2021-06-01 12:00:00.000 : 248400 seconds
2021-06-01 12:00:00.000 : 259200 seconds
2021-06-01 12:00:00.000 : 270000 seconds
2021-06-01 12:00:00.000 : 280800 seconds
2021-06-01 12:00:00.000 : 291600 seconds
2021-06-01 12:00:00.000 : 302400 seconds
2021-06-01 12:00:00.000 : 324000 seconds
2021-06-01 12:00:00.000 : 345600 seconds
2021-06-01 12:00:00.000 : 367200 seconds
2021-06-01 12:00:00.000 : 388800 seconds
2021-06-01 12:00:00.000 : 410400 seconds
2021-06-01 12:00:00.000 : 432000 seconds
2021-06-01 12:00:00.000 : 453600 seconds
2021-06-01 12:00:00.000 : 475200 seconds
2021-06-01 12:00:00.000 : 496800 seconds
2021-06-01 12:00:00.000 : 518400 seconds
2021-06-01 12:00:00.000 : 540000 seconds
2021-06-01 12:00:00.000 : 561600 seconds
2021-06-01 12:00:00.000 : 583200 seconds
2021-06-01 12:00:00.000 : 604800 seconds
2021-06-01 12:00:00.000 : 626400 seconds
2021-06-01 12:00:00.000 : 648000 seconds
2021-06-01 12:00:00.000 : 669600 seconds
2021-06-01 12:00:00.000 : 691200 seconds
2021-06-01 12:00:00.000 : 712800 seconds
2021-06-01 12:00:00.000 : 734400 seconds
2021-06-01 12:00:00.000 : 756000 seconds
2021-06-01 12:00:00.000 : 777600 seconds
2021-06-01 12:00:00.000 : 799200 seconds
2021-06-01 12:00:00.000 : 820800 seconds
2021-06-01 12:00:00.000 : 842400 seconds
2021-06-01 12:00:00.000 : 864000 seconds

Top

9 Get the Data!

Now that we have our request and DataTime fcstRun arrays ready, it’s time to request the data array from EDEX. Depending on what kind of data we’re working with, we’ll either use DataAccessLayer.getGridData() or DataAccessLayer.getGeometryData()

Note: We have more, detailed notebooks about how analyze and visualize the data once you have what you want.

## Grid Data
grid_response = DataAccessLayer.getGridData(grid_request, [grid_fcstRun[-1]])
for grid in grid_response:
    grid_data = grid.getRawData()
    lons, lats = grid.getLatLonCoords()
    print('Time :', grid.getDataTime(), "-", grid.getDataTime().getFcstTime(), 'seconds')

## Take a look at some information in our data
print('Model:', str(grid.getLocationName()))
print('Parm :', str(grid.getParameter()))
print('Unit :', str(grid.getUnit()))
print(grid_data.shape)

Time : 2021-06-01 12:00:00 - 864000 seconds
Model: GFS20
Parm : T
Unit : K
(257, 369)

Top

10 See Also

Several functions are used throughout this notebook from the DataAccessLayer class in python-awips, to see full documentation for these functions vist here.