Examples of DRILSDOWN projects

Basic Examples

• Code route to capturing movies of multiple regions in an IDV bundle
• Note: may take a few minutes, during which time if ties up your IDV session
• Code route to displaying a list of RAMADDA resouces
• See the 0:59 YouTube video intro that uses this notebook.
• Documentation is here, from this github code repo.
• Sections
• Code route: You can also get a GUI display of a set of IDV bundles using this example
• 1. Installand learn to operate the IDV: Consider my IDV intro .ppt, or Unidata’s documentation and tutorial.
• 2. Launch the IDV on this machine (from your Control Panel, or manually).
• 3. Display something meaningful in the IDV.
• The cells below capture your IDV session’s images and movies
• For exact replicability, capture a <1 MB sized IDV state bundle and embed it in this notebook
• Reading from IDV: example is here with key lines below, commented out
• Writing data into the IDV session: example is here
• using our new .to_IDV() method in xarray
• Everything in RAMADDA is an entry, with an entry type.
• A DRILSDOWN Case Study folder contains a .ipynb notebook, a .zidv or .xidv IDV bundle, and any other files.
• idv_teleport
• ramadda_publish
• Reading IDV data into this notebook as a netCDF variable
• Extract the data from .zidv bundles into Python xarray
• Examples: Loading data from python into IDV
• Example 2
• Example 3
• Vertical integral – not working
• You can call ipython_IDV without its widget Control Panel
• Then, if you want the Control Panel interface later, just call it
• See the 0:59 YouTube video intro that uses this notebook.
• Documentation is here, from this github code repo.
• Sections
• Code route: You can also get a GUI display of a set of IDV bundles using this example
• 1. Installand learn to operate the IDV: Consider my IDV intro .ppt, or Unidata’s documentation and tutorial.
• 2. Launch the IDV on this machine (from your Control Panel, or manually).
• 3. Display something meaningful in the IDV.
• The cells below capture your IDV session’s images and movies
• For exact replicability, capture a <1 MB sized IDV state bundle and embed it in this notebook
• Reading from IDV: example is here with key lines below, commented out
• Writing data into the IDV session: example is here
• using our new .to_IDV() method in xarray
• Everything in RAMADDA is an entry, with an entry type.
• A DRILSDOWN Case Study folder contains a .ipynb notebook, a .zidv or .xidv IDV bundle, and any other files.
• idv_teleport
• ramadda_publish
• Python to IDV
• IDV to Python
• Demonstrate xarray.from_zidv() method

Atmospheric (observed): What do clouds look like where they are acting to increase the water vapor?

One example of “drilling down” is to view satellite imagery around places where interesting quantitative meteorology is diagnosed. (This example)[https://github.com/Unidata/drilsdown/tree/master/UseCase_Examples/WaterVaporTendencies] constructs URLs to a NASA imagery server, from the date and time information around extreme values of a Lagrangian water vapor tendency.

The next level of scientific inquiry here is to call up a vertical sounding display, from a global 3D meteorological analysis, in either the IDV or a Jupyter Python session.

These headers within the .ipynb notebook may help clarify the workflow:

• Steps in the analysis
  • Open the daily (v4) MIMIC-AT dataset
• make custom function for drawing coastlines
• Create some tools to attach to the plots: hover tool for values
• Create some tools to attach to the plots: tap tool calls the NASA URL!
• Something about stacking so that big values can be prioritized

Atmospheric (observed): How do watervapor “islands” last long over the dry western-equatorial Indian Ocean?

Over the western equatorial Indian ocean, which is usually dry, there are occasional periods when filaments or “islands” of water vapor in the atmopshere are brought by the winds. These last surprisingly long, even as rain acts to remove the vapor. We wanted to study several such instances using vertical profile probes and column water vapor maps over region bounded by 17, 36 ; -15, 90 (North , West ; South, East).

A list of begin and end dates was supplied by by Dr. Matt Igel of UC-Davis, based on his blob-tracking results applied to column water vapor (CWV).

A script is generated to teleport the simple IDV bundle to a zidv bundle comprising of 3D data from two reanalysis for the lat-lon bounding box 17 - 36 , -15 - 90 (North - West ,South - East).

• Generate IDV_teleport script.
  • Teleport script needs a centered date and time delta to teleport, so calculate center of the state and timedelta
  • Generate teleport script by applying a function to every row of the dataframe and save it

Generated teleport bash script was run in a headless linux environment to generate zidv files. These zidv case files are then published to RAMADDA server using following command:

ramadda_publish 'Igel_WEIO*.zidv' a4154517-ac1c-4eb4-b842-572cb55ce1f2 -a 'Igel_WEIO*.gif' -username user -password pass

Links for these command line utilities: IDV_teleport , RAMADDA_publish

Atmospheric (complex, simulated): What is going on in areas where small scales are adding energy to the large-scale flow?

In this project, a global model with 7km mesh produced an enormous dataset including vertical flux of horizontal momentum (u’w’ and v’w’). The notebooks below show some “drilling down” results.

These headers within two of the .ipynb notebooks in examples/MomentumFlux_in_GlobalCloudmodel show the workflow:

• Sections
  • Simple 2-year mean map
  • Results!
• Sections
  • Geoviews is an Earth-specific Holoviews. It displays 1-2 dimensions of a multi-dimensional object, and the other dims become interactive sliders.
  • xarray is the netCDF-friendly front end for reading in the data. Cartopy has the background maps.
  • Bokeh is a plotting package with pan-zoom interactivity
• Open your 3D data set for a case, as an xarray dataset object
• Subset data in time and space if desired before lazy read
  • Use .sel( ) to select by coordinate values, or .isel( ) to select by index. Use slice(value1,value2) to select ranges.
• Merge total fields (U,V,W) with total flux (UW, VW, UV) as a Geoview (Holoview) Image object.
• Geoviews multiple-displays creation. So simple!
• G5NR_utils.py has computations with 7km gridded u,v,w inputs
  • (90,45) is the 4 degree filter scale (divides the world into 90x45 arrays)
  • Merge the prime terms, and their products, back into the Subgrid xarray dataset
• Make a Geoviews Dataset, and 6 Images, for subgrid (prime) terms
• Geoviews display for the subgrid scale filtered (“eddy”) products:
  • Resolution as a slider (or menu)
  • Should match the case selection criterion (e.g. positive SKEdot at center)
  • Bleeding edge in 10/17, needed pip install --upgrade git+https://github.com/ioam/holoviews.git
• Sections
• Open your 3D data set for a case, as an xarray dataset object
• Geoviews display for the subgrid scale filtered (“eddy”) products:
  • Set up lineplots for profile interactive
• Part of this nbviewer repo
  • Sections

Atmospheric: Weather events case studies.

• Global: GFS analyses at 1 degree (500mb Z and vorticity)
• Regional: GEOS-5 analyses at 1/4 degree and surface stations
  • Notebook from the Earthcube-supported DRILSDOWN project
• Sections within this notebook
• Loop without annotation
• Notebook from the Earthcube-supported DRILSDOWN project
  • Sections within this notebook
  • Much earlier: quiescent basin, but NH trade surge begins
• Notebook from the Earthcube-supported DRILSDOWN project
  • Background
• Sections within this notebook
  • The explanation for high heights
• The 500 mb heights
• Tell the story
• Meaning:

Oceanic: What’s going on when the Loop Current in the Gulf of Mexico splits off an eddy?

• Ocean reanalyses in 2010 (Deepwater Horizon year)
• Eddy shedding events
  • From cycle 2 of this paper’s simulations
  • Results are in