xarray with MetPy Tutorial

xarray is a powerful Python package that provides N-dimensional labeled arrays and datasets following the Common Data Model. While the process of integrating xarray features into MetPy is ongoing, this tutorial demonstrates how xarray can be used within the current version of MetPy. MetPy’s integration primarily works through accessors which allow simplified projection handling and coordinate identification. Unit and calculation support is currently available in a limited fashion, but should be improved in future versions.

import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.pyplot as plt
import xarray as xr

# Any import of metpy will activate the accessors
import metpy.calc as mpcalc
from metpy.cbook import get_test_data
from metpy.units import units
from metpy.xarray import grid_deltas_from_dataarray

Getting Data

While xarray can handle a wide variety of n-dimensional data (essentially anything that can be stored in a netCDF file), a common use case is working with model output. Such model data can be obtained from a THREDDS Data Server using the siphon package, but for this tutorial, we will use an example subset of GFS data from Hurrican Irma (September 5th, 2017).

# Open the netCDF file as a xarray Dataset
data = xr.open_dataset(get_test_data('irma_gfs_example.nc', False))

# View a summary of the Dataset
print(data)

Out:

<xarray.Dataset>
Dimensions:                              (isobaric1: 21, isobaric3: 31, latitude: 81, longitude: 131, time1: 9)
Coordinates:
  * time1                                (time1) datetime64[ns] 2017-09-05T12...
    reftime                              datetime64[ns] ...
  * latitude                             (latitude) float32 50.0 49.5 ... 10.0
  * isobaric3                            (isobaric3) float64 100.0 ... 1e+05
  * isobaric1                            (isobaric1) float64 1e+04 ... 1e+05
  * longitude                            (longitude) float32 250.0 ... 315.0
Data variables:
    Vertical_velocity_pressure_isobaric  (time1, isobaric1, latitude, longitude) float32 ...
    Relative_humidity_isobaric           (time1, isobaric3, latitude, longitude) float32 ...
    Temperature_isobaric                 (time1, isobaric3, latitude, longitude) float32 ...
    u-component_of_wind_isobaric         (time1, isobaric3, latitude, longitude) float32 ...
    v-component_of_wind_isobaric         (time1, isobaric3, latitude, longitude) float32 ...
    Geopotential_height_isobaric         (time1, isobaric3, latitude, longitude) float32 ...
    LatLon_361X720-0p25S-180p00E         int32 ...
Attributes:
    Originating_or_generating_Center:                                        ...
    Originating_or_generating_Subcenter:                                     ...
    GRIB_table_version:                                                      ...
    Type_of_generating_process:                                              ...
    Analysis_or_forecast_generating_process_identifier_defined_by_originating...
    Conventions:                                                             ...
    history:                                                                 ...
    featureType:                                                             ...
    History:                                                                 ...
    geospatial_lat_min:                                                      ...
    geospatial_lat_max:                                                      ...
    geospatial_lon_min:                                                      ...
    geospatial_lon_max:                                                      ...

Preparing Data

To make use of the data within MetPy, we need to parse the dataset for projection information following the CF conventions. For this, we use the data.metpy.parse_cf() method, which will return a new, parsed DataArray or Dataset.

Additionally, we rename our data variables for easier reference.

# To parse the full dataset, we can call parse_cf without an argument, and assign the returned
# Dataset.
data = data.metpy.parse_cf()

# If we instead want just a single variable, we can pass that variable name to parse_cf and
# it will return just that data variable as a DataArray.
data_var = data.metpy.parse_cf('Temperature_isobaric')

# If we want only a subset of variables, we can pass a list of variable names as well.
data_subset = data.metpy.parse_cf(['u-component_of_wind_isobaric',
                                   'v-component_of_wind_isobaric'])

# To rename variables, supply a dictionary between old and new names to the rename method
data = data.rename({
    'Vertical_velocity_pressure_isobaric': 'omega',
    'Relative_humidity_isobaric': 'relative_humidity',
    'Temperature_isobaric': 'temperature',
    'u-component_of_wind_isobaric': 'u',
    'v-component_of_wind_isobaric': 'v',
    'Geopotential_height_isobaric': 'height'
})

Units

MetPy’s DataArray accessor has a unit_array property to obtain a pint.Quantity array of just the data from the DataArray (metadata is removed) and a convert_units method to convert the the data from one unit to another (keeping it as a DataArray). For now, we’ll just use convert_units to convert our temperature to degC.

data['temperature'] = data['temperature'].metpy.convert_units('degC')

Coordinates

You may have noticed how we directly accessed the vertical coordinates above using their names. However, in general, if we are working with a particular DataArray, we don’t have to worry about that since MetPy is able to parse the coordinates and so obtain a particular coordinate type directly. There are two ways to do this:

1. Use the data_var.metpy.coordinates method

2. Use the data_var.metpy.x, data_var.metpy.y, data_var.metpy.longitude, data_var.metpy.latitude, data_var.metpy.vertical, data_var.metpy.time properties

The valid coordinate types are:

• x

• y

• longitude

• latitude

• vertical

• time

(Both approaches are shown below)

The x, y, vertical, and time coordinates cannot be multidimensional, however, the longitude and latitude coordinates can (which is often the case for gridded weather data in its native projection). Note that for gridded data on an equirectangular projection, such as the GFS data in this example, x and longitude will be identical (as will y and latitude).

# Get multiple coordinates (for example, in just the x and y direction)
x, y = data['temperature'].metpy.coordinates('x', 'y')

# If we want to get just a single coordinate from the coordinates method, we have to use
# tuple unpacking because the coordinates method returns a generator
vertical, = data['temperature'].metpy.coordinates('vertical')

# Or, we can just get a coordinate from the property
time = data['temperature'].metpy.time

# To verify, we can inspect all their names
print([coord.name for coord in (x, y, vertical, time)])

Out:

['longitude', 'latitude', 'isobaric3', 'time1']

Indexing and Selecting Data

MetPy provides wrappers for the usual xarray indexing and selection routines that can handle quantities with units. For DataArrays, MetPy also allows using the coordinate axis types mentioned above as aliases for the coordinates. And so, if we wanted 850 hPa heights, we would take:

print(data['height'].metpy.sel(vertical=850 * units.hPa))

Out:

<xarray.DataArray 'height' (time1: 9, latitude: 81, longitude: 131)>
array([[[1593.8025, 1594.1064, ..., 1453.2264, 1458.3625],
        [1594.7784, 1595.0825, ..., 1463.3065, 1468.4905],
        ...,
        [1511.4985, 1511.8025, ..., 1537.6584, 1536.9385],
        [1511.2745, 1511.9305, ..., 1537.9146, 1537.0825]],

       [[1590.0658, 1590.8018, ..., 1463.2977, 1465.3617],
        [1591.5537, 1591.9377, ..., 1472.6577, 1475.2338],
        ...,
        [1522.8977, 1523.2977, ..., 1536.4337, 1535.5857],
        [1523.0417, 1523.4738, ..., 1536.7058, 1535.6978]],

       ...,

       [[1546.7413, 1547.8773, ..., 1553.7174, 1555.9254],
        [1547.8453, 1549.2374, ..., 1563.6373, 1565.6533],
        ...,
        [1514.6774, 1514.1814, ..., 1524.7733, 1524.3093],
        [1515.4934, 1515.0294, ..., 1524.8214, 1524.3414]],

       [[1542.1625, 1542.6906, ..., 1561.7946, 1564.2264],
        [1542.2426, 1543.2345, ..., 1571.6665, 1574.0825],
        ...,
        [1507.6505, 1507.2025, ..., 1538.3385, 1537.8905],
        [1508.5625, 1508.0505, ..., 1538.8185, 1538.6105]]], dtype=float32)
Coordinates:
  * time1      (time1) datetime64[ns] 2017-09-05T12:00:00 ... 2017-09-06T12:0...
    reftime    datetime64[ns] 2017-09-05T12:00:00
  * latitude   (latitude) float32 50.0 49.5 49.0 48.5 ... 11.5 11.0 10.5 10.0
  * longitude  (longitude) float32 250.0 250.5 251.0 251.5 ... 314.0 314.5 315.0
    metpy_crs  object Projection: latitude_longitude
    isobaric3  float64 8.5e+04
Attributes:
    long_name:                      Geopotential height @ Isobaric surface
    units:                          gpm
    Grib_Variable_Id:               VAR_0-3-5_L100
    Grib2_Parameter:                [0 3 5]
    Grib2_Parameter_Discipline:     Meteorological products
    Grib2_Parameter_Category:       Mass
    Grib2_Parameter_Name:           Geopotential height
    Grib2_Level_Type:               100
    Grib2_Level_Desc:               Isobaric surface
    Grib2_Generating_Process_Type:  Forecast
    grid_mapping:                   LatLon_361X720-0p25S-180p00E

For full details on xarray indexing/selection, see xarray’s documentation.

Projections

Getting the cartopy coordinate reference system (CRS) of the projection of a DataArray is as straightforward as using the data_var.metpy.cartopy_crs property:

cartopy_crs = data['temperature'].metpy.cartopy_crs
print(cartopy_crs)

Out:

<cartopy.crs.PlateCarree object at 0x7f1ad4af8ef0>

Likewise, the PyProj CRS can be obtained with the .pyproj_crs property:

pyproj_crs = data['temperature'].metpy.pyproj_crs
print(pyproj_crs)

Out:

{"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "GeographicCRS", "name": "undefined", "datum": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "GeodeticReferenceFrame", "name": "undefined", "ellipsoid": {"name": "undefined", "radius": 6371229}}, "coordinate_system": {"$schema": "https://proj.org/schemas/v0.2/projjson.schema.json", "type": "CoordinateSystem", "subtype": "ellipsoidal", "axis": [{"name": "Longitude", "abbreviation": "lon", "direction": "east", "unit": "degree"}, {"name": "Latitude", "abbreviation": "lat", "direction": "north", "unit": "degree"}]}}

The cartopy Globe can similarly be accessed via the data_var.metpy.cartopy_globe property:

cartopy_globe = data['temperature'].metpy.cartopy_globe
print(cartopy_globe)

Out:

<cartopy._crs.Globe object at 0x7f1ad4b07580>

Calculations

TODO: The below section is out of date as of PR #1490. Updates are needed as a part of the 1.0 Upgrade Guide and Doc Update (Issue #1385).

Most of the calculations in metpy.calc will accept DataArrays by converting them into their corresponding unit arrays. While this may often work without any issues, we must keep in mind that because the calculations are working with unit arrays and not DataArrays:

• The calculations will return unit arrays rather than DataArrays

• Broadcasting must be taken care of outside of the calculation, as it would only recognize dimensions by order, not name

As an example, we calculate geostropic wind at 500 hPa below:

lat, lon = xr.broadcast(y, x)
dx, dy = mpcalc.lat_lon_grid_deltas(lon, lat, geod=pyproj_crs.get_geod())
heights = data['height'].metpy.loc[{'time': time[0], 'vertical': 500. * units.hPa}]
u_geo, v_geo = mpcalc.geostrophic_wind(heights, dx, dy, lat)
print(u_geo)
print(v_geo)

Out:

<xarray.DataArray (latitude: 81, longitude: 131)>
<Quantity([[ 2.2733034   6.25293339  3.28353199 ... 33.12285541 32.74320064
  32.58517175]
 [ 3.81693346  4.38966762  4.32598735 ... 31.0138463  30.7595135
  30.53663254]
 [ 5.1277875   4.83945459  5.80108591 ... 29.19673862 29.03672755
  28.58760112]
 ...
 [-1.90177703 -1.14044725 -1.90177703 ... -6.97111921 -7.35333151
  -6.84577833]
 [-2.78677553 -2.65553785 -2.38982204 ... -6.37069849 -7.16622569
  -6.90050988]
 [-7.10062943 -8.36058307 -3.62002875 ... -5.57712273 -7.51721329
  -8.3537817 ]], 'meter / second')>
Coordinates:
    time1      datetime64[ns] 2017-09-05T12:00:00
    reftime    datetime64[ns] 2017-09-05T12:00:00
  * latitude   (latitude) float32 50.0 49.5 49.0 48.5 ... 11.5 11.0 10.5 10.0
  * longitude  (longitude) float32 250.0 250.5 251.0 251.5 ... 314.0 314.5 315.0
    metpy_crs  object Projection: latitude_longitude
    isobaric3  float64 5e+04
<xarray.DataArray (latitude: 81, longitude: 131)>
<Quantity([[-14.83673367 -14.44217477 -16.94684424 ...   5.599978     6.53360749
    7.41866665]
 [ -8.32614256 -13.81233272 -17.14255058 ...   5.28948608   6.12233949
    7.20092352]
 [-12.01837752 -13.58075467 -14.89923401 ...   4.9340109    5.76469463
    6.74147851]
 ...
 [  4.26253418   3.48695474   2.96990178 ...   3.22842826   3.22842826
    1.93579585]
 [  6.61761      3.10447641   2.02351222 ...   2.42887379   3.64331069
    3.37636526]
 [  4.80505261   4.24218982   2.54669515 ...   1.55736883   2.9714321
    3.81745286]], 'meter / second')>
Coordinates:
    time1      datetime64[ns] 2017-09-05T12:00:00
    reftime    datetime64[ns] 2017-09-05T12:00:00
  * latitude   (latitude) float32 50.0 49.5 49.0 48.5 ... 11.5 11.0 10.5 10.0
  * longitude  (longitude) float32 250.0 250.5 251.0 251.5 ... 314.0 314.5 315.0
    metpy_crs  object Projection: latitude_longitude
    isobaric3  float64 5e+04

Also, a limited number of calculations directly support xarray DataArrays or Datasets (they can accept and return xarray objects). Right now, this includes

• Derivative functions

  • first_derivative

  • second_derivative

  • gradient

  • laplacian

• Cross-section functions

  • cross_section_components

  • normal_component

  • tangential_component

  • absolute_momentum

• Smoothing functions

  • smooth_gaussian

  • smooth_n_point

  • smooth_window

  • smooth_rectangular

  • smooth_circular

More details can be found by looking at the documentation for the specific function of interest.

There is also the special case of the helper function, grid_deltas_from_dataarray, which takes a DataArray input, but returns unit arrays for use in other calculations. We could rewrite the above geostrophic wind example using this helper function as follows:

heights = data['height'].metpy.loc[{'time': time[0], 'vertical': 500. * units.hPa}]
lat, lon = xr.broadcast(y, x)
dx, dy = grid_deltas_from_dataarray(heights)
u_geo, v_geo = mpcalc.geostrophic_wind(heights, dx, dy, lat)
print(u_geo)
print(v_geo)

Out:

<xarray.DataArray (latitude: 81, longitude: 131)>
<Quantity([[ 2.2733034   6.25293339  3.28353199 ... 33.12285541 32.74320064
  32.58517175]
 [ 3.81693346  4.38966762  4.32598735 ... 31.0138463  30.7595135
  30.53663254]
 [ 5.1277875   4.83945459  5.80108591 ... 29.19673862 29.03672755
  28.58760112]
 ...
 [-1.90177703 -1.14044725 -1.90177703 ... -6.97111921 -7.35333151
  -6.84577833]
 [-2.78677553 -2.65553785 -2.38982204 ... -6.37069849 -7.16622569
  -6.90050988]
 [-7.10062943 -8.36058307 -3.62002875 ... -5.57712273 -7.51721329
  -8.3537817 ]], 'meter / second')>
Coordinates:
    time1      datetime64[ns] 2017-09-05T12:00:00
    reftime    datetime64[ns] 2017-09-05T12:00:00
  * latitude   (latitude) float32 50.0 49.5 49.0 48.5 ... 11.5 11.0 10.5 10.0
  * longitude  (longitude) float32 250.0 250.5 251.0 251.5 ... 314.0 314.5 315.0
    metpy_crs  object Projection: latitude_longitude
    isobaric3  float64 5e+04
<xarray.DataArray (latitude: 81, longitude: 131)>
<Quantity([[-14.83673367 -14.44217477 -16.94684424 ...   5.599978     6.53360749
    7.41866665]
 [ -8.32614256 -13.81233272 -17.14255058 ...   5.28948608   6.12233949
    7.20092352]
 [-12.01837752 -13.58075467 -14.89923401 ...   4.9340109    5.76469463
    6.74147851]
 ...
 [  4.26253418   3.48695474   2.96990178 ...   3.22842826   3.22842826
    1.93579585]
 [  6.61761      3.10447641   2.02351222 ...   2.42887379   3.64331069
    3.37636526]
 [  4.80505261   4.24218982   2.54669515 ...   1.55736883   2.9714321
    3.81745286]], 'meter / second')>
Coordinates:
    time1      datetime64[ns] 2017-09-05T12:00:00
    reftime    datetime64[ns] 2017-09-05T12:00:00
  * latitude   (latitude) float32 50.0 49.5 49.0 48.5 ... 11.5 11.0 10.5 10.0
  * longitude  (longitude) float32 250.0 250.5 251.0 251.5 ... 314.0 314.5 315.0
    metpy_crs  object Projection: latitude_longitude
    isobaric3  float64 5e+04

Plotting

Like most meteorological data, we want to be able to plot these data. DataArrays can be used like normal numpy arrays in plotting code, which is the recommended process at the current point in time, or we can use some of xarray’s plotting functionality for quick inspection of the data.

(More detail beyond the following can be found at xarray’s plotting reference.)

# A very simple example example of a plot of 500 hPa heights
data['height'].metpy.loc[{'time': time[0], 'vertical': 500. * units.hPa}].plot()
plt.show()

# Let's add a projection and coastlines to it
ax = plt.axes(projection=ccrs.LambertConformal())
data['height'].metpy.loc[{'time': time[0],
                          'vertical': 500. * units.hPa}].plot(ax=ax, transform=cartopy_crs)
ax.coastlines()
plt.show()

# Or, let's make a full 500 hPa map with heights, temperature, winds, and humidity

# Select the data for this time and level
data_level = data.metpy.loc[{time.name: time[0], vertical.name: 500. * units.hPa}]

# Create the matplotlib figure and axis
fig, ax = plt.subplots(1, 1, figsize=(12, 8), subplot_kw={'projection': cartopy_crs})

# Plot RH as filled contours
rh = ax.contourf(x, y, data_level['relative_humidity'], levels=[70, 80, 90, 100],
                 colors=['#99ff00', '#00ff00', '#00cc00'])

# Plot wind barbs, but not all of them
wind_slice = slice(5, -5, 5)
ax.barbs(x[wind_slice], y[wind_slice],
         data_level['u'].metpy.unit_array[wind_slice, wind_slice].to('knots'),
         data_level['v'].metpy.unit_array[wind_slice, wind_slice].to('knots'),
         length=6)

# Plot heights and temperature as contours
h_contour = ax.contour(x, y, data_level['height'], colors='k', levels=range(5400, 6000, 60))
h_contour.clabel(fontsize=8, colors='k', inline=1, inline_spacing=8,
                 fmt='%i', rightside_up=True, use_clabeltext=True)
t_contour = ax.contour(x, y, data_level['temperature'], colors='xkcd:deep blue',
                       levels=range(-26, 4, 2), alpha=0.8, linestyles='--')
t_contour.clabel(fontsize=8, colors='xkcd:deep blue', inline=1, inline_spacing=8,
                 fmt='%i', rightside_up=True, use_clabeltext=True)

# Add geographic features
ax.add_feature(cfeature.LAND.with_scale('50m'), facecolor=cfeature.COLORS['land'])
ax.add_feature(cfeature.OCEAN.with_scale('50m'), facecolor=cfeature.COLORS['water'])
ax.add_feature(cfeature.STATES.with_scale('50m'), edgecolor='#c7c783', zorder=0)
ax.add_feature(cfeature.LAKES.with_scale('50m'), facecolor=cfeature.COLORS['water'],
               edgecolor='#c7c783', zorder=0)

# Set a title and show the plot
ax.set_title('500 hPa Heights (m), Temperature (\u00B0C), Humidity (%) at '
             + time[0].dt.strftime('%Y-%m-%d %H:%MZ').item())
plt.show()

What Could Go Wrong?

Depending on your dataset and what you are trying to do, you might run into problems with xarray and MetPy. Below are examples of some of the most common issues

• Multiple coordinate conflict

• An axis not being available

• An axis not being interpretable

• Arrays not broadcasting in calculations

Coordinate Conflict

Code:

x = data['Temperature'].metpy.x

Error Message:

/home/user/env/MetPy/metpy/xarray.py:305: UserWarning: More than
one x coordinate present for variable "Temperature".

Fix:

Manually assign the coordinates using the assign_coordinates() method on your DataArray, or by specifying the coordinates argument to the parse_cf() method on your Dataset, to map the time, vertical, y, latitude, x, and longitude axes (as applicable to your data) to the corresponding coordinates.

data['Temperature'].assign_coordinates({'time': 'time', 'vertical': 'isobaric',
                                        'y': 'y', 'x': 'x'})
x = data['Temperature'].metpy.x

or

temperature = data.metpy.parse_cf('Temperature',
                                  coordinates={'time': 'time', 'vertical': 'isobaric',
                                               'y': 'y', 'x': 'x'})
x = temperature.metpy.x

Axis Unavailable

Code:

data['Temperature'].metpy.vertical

Error Message:

AttributeError: vertical attribute is not available.

This means that your data variable does not have the coordinate that was requested, at least as far as the parser can recognize. Verify that you are requesting a coordinate that your data actually has, and if it still is not available, you will need to manually specify the coordinates as discussed above.

Axis Not Interpretable

Code:

x, y, ensemble = data['Temperature'].metpy.coordinates('x', 'y', 'ensemble')

Error Message:

AttributeError: 'ensemble' is not an interpretable axis

This means that you are requesting a coordinate that MetPy is (currently) unable to parse. While this means it cannot be recognized automatically, you can still obtain your desired coordinate directly by accessing it by name. If you have a need for systematic identification of a new coordinate type, we welcome pull requests for such new functionality on GitHub!

Broadcasting in Calculations

Code:

theta = mpcalc.potential_temperature(data['isobaric3'], data['temperature'])

Error Message:

ValueError: operands could not be broadcast together with shapes (9,31,81,131) (31,)

This is a symptom of the incomplete integration of xarray with MetPy’s calculations; the calculations currently convert the DataArrays to unit arrays that do not recognize which coordinates match with which. And so, we must do some manipulations.

Fix 1 (xarray broadcasting):

pressure, temperature = xr.broadcast(data['isobaric3'], data['temperature'])
theta = mpcalc.potential_temperature(pressure, temperature)

Fix 2 (unit array broadcasting):

theta = mpcalc.potential_temperature(
    data['isobaric3'].metpy.unit_array[None, :, None, None],
    data['temperature'].metpy.unit_array
)