Numpy and arrays

We do not advise you to use Numpy directly. The XArray and Pandas packages are now mature enough to be used instead. However, Numpy is the historical package for numerical calculations in Python. The other packages have been built upon it, i.e. they encapsulate Numpy arrays with additional information to simplify some analyses.

Additionally, some functionalities of Numpy haven't been ported to Xarray and should be used directly. Additionally, most examples you will find in documentation of some packages (e.g. SciPy) or StackOverflow might use Numpy. It is worth for you knowing about this package and understanding how to adapt Numpy examples to Xarray.

In [1]:
import numpy as np

Create an array

There are plenty of ways to do so depending on what you want. The low-level function is np.ndarray() which you probably won't use much. But the webpage for this function is interesting as it lists all attributes and methods associated with numpy arrays!

In [2]:
# Low-level 
rr = np.ndarray(shape=(2,4,3),dtype=float)
rr
Out[2]:
array([[[2.00000000e+000, 2.00000000e+000, 9.38724727e-323],
        [0.00000000e+000, 0.00000000e+000, 5.02034658e+175],
        [1.21647076e-046, 4.95419814e-062, 3.27611965e+179],
        [1.51954764e-051, 5.82777938e-144, 1.12855837e+277]],

       [[6.32672840e+180, 2.66371964e+233, 5.04621361e+180],
        [8.37170571e-144, 1.36347154e+161, 4.50618609e-144],
        [7.79952704e-143, 8.45341552e+169, 1.68749783e+160],
        [1.16066661e-046, 1.23139611e+165, 1.41529403e+161]]])
In [3]:
# From list or tuple
rr = np.array([[3.,4.],[5.,6.]])
print(rr)
[[3. 4.]
 [5. 6.]]
In [4]:
rr1 = np.array([['Claire','Paola'],['Scott','Danny']])  # It doesn't have to be a numerical type  
print(rr1)
[['Claire' 'Paola']
 ['Scott' 'Danny']]
In [5]:
rr2 = np.array([['Claire',10], ['Paola', 6]]) # It doesn't have to be only 1 type
print(rr2)
[['Claire' '10']
 ['Paola' '6']]
In [6]:
# Initialise to 0 or 1.
rr = np.zeros((2,3),dtype=float)
print(rr)
rr1 = np.ones((2,4),dtype=np.int32)
print(rr1)
[[0. 0. 0.]
 [0. 0. 0.]]
[[1 1 1 1]
 [1 1 1 1]]
In [7]:
# Same shape as an existing array. It's possible to choose the data-type with the dtype argument.
rr2 = np.zeros_like(rr1)
rr2
Out[7]:
array([[0, 0, 0, 0],
       [0, 0, 0, 0]], dtype=int32)
In [8]:
# Evenly spaced values
rr2= np.arange(5,45,2)
rr2
Out[8]:
array([ 5,  7,  9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37,
       39, 41, 43])
In [9]:
# Reshaping an existing array
rr2 = rr2.reshape((5,2,2))
rr2
Out[9]:
array([[[ 5,  7],
        [ 9, 11]],

       [[13, 15],
        [17, 19]],

       [[21, 23],
        [25, 27]],

       [[29, 31],
        [33, 35]],

       [[37, 39],
        [41, 43]]])

Read data from file

Do you remember the csv example from the last training? Here it is with numpy.

In [10]:
li = np.loadtxt('test.txt',delimiter=',',skiprows=2)
print(li)
# For the third format example, simply take the transpose
print(li.T)
# You want the columns in separate arrays?
c1,c2,c3 = np.loadtxt('test.txt', delimiter=',',skiprows=2,unpack=True)
print(c1,c2,c3)
[[50. 30. 40.]
 [70. 20. 30.]]
[[50. 70.]
 [30. 20.]
 [40. 30.]]
[50. 70.] [30. 20.] [40. 30.]

Indexing

It is the same as for lists etc, except for the multi-dimensional part:

In [14]:
print(f"First element {rr2[0,0,0]}\n")
print(f"First index of the second dimension \n {rr2[:,0,:]}\n")
print(f"First 2 indexes along the 1st dimension and all other indexes along other dimensions\n {rr2[:2,:,:]}\n")
print(f"Stride {rr2[0:5:2,0,1]}\n")
First element 5

First index of the second dimension 
 [[ 5  7]
 [13 15]
 [21 23]
 [29 31]
 [37 39]]

First 2 indexes along the 1st dimension and all other indexes along other dimensions
 [[[ 5  7]
  [ 9 11]]

 [[13 15]
  [17 19]]]

Stride [ 7 23 39]

There is a generic form to say "all other indexes along all other dimensions", i.e. "everything else", without specifying the number of dimensions in your array. It can be used to indicate all dimensions before or after the specified slice:

In [20]:
print(f"Specify slices in all dimensions: \n{rr2[:2,:,:]}\n")
print(f"Generic form:\n{rr2[:2,...]}\n")
print(f"Any number of dimensions specified before:\n{rr2[:2,0,...]}\n")
print(f"Works for the start of the array as well:\n{rr2[...,0]}\n")
print(f"You can omit the dimensions, but it is confusing to read:\n{rr2[:2]}\n")
Specify slices in all dimensions: 
[[[ 5  7]
  [ 9 11]]

 [[13 15]
  [17 19]]]

Generic form:
[[[ 5  7]
  [ 9 11]]

 [[13 15]
  [17 19]]]

Any number of dimensions specified before:
[[ 5  7]
 [13 15]]

Works for the start of the array as well:
[[ 5  9]
 [13 17]
 [21 25]
 [29 33]
 [37 41]]

You can omit the dimensions, but it is confusing to read:
[[[ 5  7]
  [ 9 11]]

 [[13 15]
  [17 19]]]

Matlab users

In Matlab, arrays are matrices. That is not true in Python. This means in Matlab, the multiplication is the matrice multiplication, in Python that's multiplication element by element. This page provides a long table of equivalents between Matlab and Python.

Operations with arrays along some given axis

numpy has a lot of handy functions for common operations. For example if you want the mean of an array:

In [21]:
rr2.mean()
Out[21]:
24.0

That's handy, what is even more handy is the possibility to calculate the mean over a given dimension only. For example, rr2 is 3D. Let's say the dimensions are time, latitude and longitude respectively and you want to calculate the time average at each spatial point:

In [22]:
rr2.mean(axis=0)  # Remember indexes start at 0
Out[22]:
array([[21., 23.],
       [25., 27.]])

Working with time

There is already a lot out there. You probably won't need to develop much yourself.

Numpy has a date data type: datetime64. Do not confuse datetime64 from Numpy and datetime from Python! They do not have the same methods or abilities. Both can be useful.

datetime64 is relatively simple, it doesn't have a lot of built-in capabilities. When doing fancy date calculations in Python, the must is probably to work with pandas. pandas is built upon numpy so can readily convert datetime64 to its own date and time objects.

Note xarray and pandas are also very compatible with each other.

In [23]:
print(np.datetime64('2020-04-15T05:00','ms'))
print(np.datetime64('2020-04-10','M'))
2020-04-15T05:00:00.000
2020-04

You can do simple calculations. For example, the number of days in February 2036:

In [24]:
print(np.datetime64('2036-03','D')-np.datetime64('2036-02','D'))
29 days

Be careful of the unit:

In [25]:
print(np.datetime64('2036-03','M')-np.datetime64('2036-02','M'))
1 months

It's possible to convert units. So if you want both the number of months and the number of days:

In [26]:
timeA = np.datetime64('1988-05','M')
timeB = np.datetime64('1990-03','M')
delta = timeB - timeA
print(delta)
deltaD = timeB.astype('datetime64[D]') - timeA.astype('datetime64[D]')
print(deltaD)
22 months
669 days

Note, delta and deltaD are not the strings printed out above. They are numpy.timedelta64 objects. The print() function gives a pretty output because of the way the object has been developed.

In [27]:
deltaD
Out[27]:
numpy.timedelta64(669,'D')

Xarray

Numpy is great but it is very generic. And it only gives you the raw data. The coder has to keep track of the additional information: is there a time dimension? What field does this data represent? etc.

Xarray introduces labelled arrays which typically means you get self-described arrays: name of the field, name of dimensions, coordinates for the dimensions, etc.

As such it works very well with the netCDF format since this is also a self-describing format.

In [28]:
import xarray as xr

Reading in netcdf file

In [32]:
# open netcdf file
ds = xr.open_dataset("http://dapds00.nci.org.au/thredds/dodsC/ks32/ARCCSS_Data/ACCESS1-3b_AMZDEF/v1-0/001GPwc.tas_pr_monthly_TS.1978_2011.nc")
# see how all the info is there
ds
# print just a variable to see the variable level attributes.
Out[32]:
Show/Hide data repr Show/Hide attributes
xarray.Dataset
    • lat: 145
    • lon: 192
    • time: 408
    • lat
      (lat)
      float32
      -90.0 -88.75 -87.5 ... 88.75 90.0
      units :
      degrees_north
      standard_name :
      latitude
      point_spacing :
      even
      axis :
      Y
      array([-90.  , -88.75, -87.5 , -86.25, -85.  , -83.75, -82.5 , -81.25, -80.  ,
             -78.75, -77.5 , -76.25, -75.  , -73.75, -72.5 , -71.25, -70.  , -68.75,
             -67.5 , -66.25, -65.  , -63.75, -62.5 , -61.25, -60.  , -58.75, -57.5 ,
             -56.25, -55.  , -53.75, -52.5 , -51.25, -50.  , -48.75, -47.5 , -46.25,
             -45.  , -43.75, -42.5 , -41.25, -40.  , -38.75, -37.5 , -36.25, -35.  ,
             -33.75, -32.5 , -31.25, -30.  , -28.75, -27.5 , -26.25, -25.  , -23.75,
             -22.5 , -21.25, -20.  , -18.75, -17.5 , -16.25, -15.  , -13.75, -12.5 ,
             -11.25, -10.  ,  -8.75,  -7.5 ,  -6.25,  -5.  ,  -3.75,  -2.5 ,  -1.25,
               0.  ,   1.25,   2.5 ,   3.75,   5.  ,   6.25,   7.5 ,   8.75,  10.  ,
              11.25,  12.5 ,  13.75,  15.  ,  16.25,  17.5 ,  18.75,  20.  ,  21.25,
              22.5 ,  23.75,  25.  ,  26.25,  27.5 ,  28.75,  30.  ,  31.25,  32.5 ,
              33.75,  35.  ,  36.25,  37.5 ,  38.75,  40.  ,  41.25,  42.5 ,  43.75,
              45.  ,  46.25,  47.5 ,  48.75,  50.  ,  51.25,  52.5 ,  53.75,  55.  ,
              56.25,  57.5 ,  58.75,  60.  ,  61.25,  62.5 ,  63.75,  65.  ,  66.25,
              67.5 ,  68.75,  70.  ,  71.25,  72.5 ,  73.75,  75.  ,  76.25,  77.5 ,
              78.75,  80.  ,  81.25,  82.5 ,  83.75,  85.  ,  86.25,  87.5 ,  88.75,
              90.  ], dtype=float32)
    • lon
      (lon)
      float32
      0.0 1.875 3.75 ... 356.25 358.125
      units :
      degrees_east
      standard_name :
      longitude
      modulo :
      360.0
      point_spacing :
      even
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125],
            dtype=float32)
    • time
      (time)
      datetime64[ns]
      1978-01-16T12:00:00 ... 2011-12-16T12:00:00
      standard_name :
      time
      axis :
      T
      array(['1978-01-16T12:00:00.000000000', '1978-02-15T00:00:00.000000000',
             '1978-03-16T12:00:00.000000000', ..., '2011-10-16T12:00:00.000000000',
             '2011-11-16T00:00:00.000000000', '2011-12-16T12:00:00.000000000'],
            dtype='datetime64[ns]')
    • pr
      (time, lat, lon)
      float32
      ...
      stash_item :
      216
      stash_model :
      1
      lookup_source :
      defaults (cdunifpp V0.13)
      long_name :
      total precipitation rate (kg/m2/s)
      cell_methods :
      time: mean
      units :
      kg m-2 s-1
      stash_section :
      5
      name :
      pr
      standard_name :
      precipitation_flux
      [11358720 values with dtype=float32]
    • tas
      (time, lat, lon)
      float32
      ...
      stash_item :
      236
      stash_model :
      1
      lookup_source :
      defaults (cdunifpp V0.13)
      long_name :
      air temperature at 1.5 m
      cell_methods :
      time: mean
      units :
      K
      stash_section :
      3
      name :
      tas
      standard_name :
      air_temperature
      [11358720 values with dtype=float32]
  • input_file_format :
    UM ancillary
    input_uri :
    file:///short/dt6/rzl561/UM_ROUTDIR/rzl561/uaoyd/uaoyda.pah8jan
    input_word_length :
    8
    history :
    Wed Jun 4 11:09:52 2014: ncks -v tas,pr uaoyd.monthly_TS.1978_2011.nc uaoyd.tas_pr_monthly_TS.1978_2011.nc PP/UM file read by cdunif. Converted to NetCDF by rzl561 on 2014-01-25. Mar 2014 ncrcat -O uaoyd.monthly.????.nc uaoyd.monthly_TS.1978_2011.nc Mar 2014 ncrcat -O uaoyd.monthly.YEAR-??-00T00.nc uaoyd.monthly.YEAR.nc
    input_byte_ordering :
    big_endian
    nco_openmp_thread_number :
    1
    NCO :
    4.3.8
    summary :
    ACCESS1.3b model output from Land use change experiments gradual increasing area of forest pft changed to grass
    institution :
    ARC Centre of Excellence for Climate System Science, model run at NCI National Facility
    source :
    ACCESS1.3b (HadGEM3-CABLE2.0), AMIP, N96
    contact :
    [email protected]
    title :
    ACCESS1.3b model output from the Amazonian Deforestation AMZDEF experiment v1.0
    Conventions :
    CF-1.6, ACDD-1.3
    license :
    http://creativecommons.org/licenses/by-nc-nd/4.0/
    id :
    http://dx.doi.org/10.4225/41/563BEB78E3A93
    keywords :
    climatology
    date_created :
    2014-06-04
    creator_name :
    Ruth Lorenz
    creator_email :
    [email protected]
    publisher_name :
    ARCCSS data manager
    publisher_email :
    [email protected]
    product_version :
    1.0
    references :
    Lorenz, R., and A. J. Pitman (2014), Effect of land-atmosphere coupling strength on impacts from Amazonian deforestation, Geophys. Res. Lett., 41, 5987-5995, doi:10.1002/2014GL061017
    time_coverage_start :
    1978-01-01T00:00
    time_coverage_end :
    2012-01-01T00:00
    geospatial_lat_min :
    -90.0
    geospatial_lat_max :
    90.0
    geospatial_lon_min :
    0.0
    geospatial_lon_max :
    358.125
    DODS_EXTRA.Unlimited_Dimension :
    time
In [33]:
# Variables are stored in the Dataset in a dictionary so you can refer to them by name
tas = ds['tas']
tas
Out[33]:
Show/Hide data repr Show/Hide attributes
xarray.DataArray
'tas'
  • time: 408
  • lat: 145
  • lon: 192
  • ...
    [11358720 values with dtype=float32]
    • lat
      (lat)
      float32
      -90.0 -88.75 -87.5 ... 88.75 90.0
      units :
      degrees_north
      standard_name :
      latitude
      point_spacing :
      even
      axis :
      Y
      array([-90.  , -88.75, -87.5 , -86.25, -85.  , -83.75, -82.5 , -81.25, -80.  ,
             -78.75, -77.5 , -76.25, -75.  , -73.75, -72.5 , -71.25, -70.  , -68.75,
             -67.5 , -66.25, -65.  , -63.75, -62.5 , -61.25, -60.  , -58.75, -57.5 ,
             -56.25, -55.  , -53.75, -52.5 , -51.25, -50.  , -48.75, -47.5 , -46.25,
             -45.  , -43.75, -42.5 , -41.25, -40.  , -38.75, -37.5 , -36.25, -35.  ,
             -33.75, -32.5 , -31.25, -30.  , -28.75, -27.5 , -26.25, -25.  , -23.75,
             -22.5 , -21.25, -20.  , -18.75, -17.5 , -16.25, -15.  , -13.75, -12.5 ,
             -11.25, -10.  ,  -8.75,  -7.5 ,  -6.25,  -5.  ,  -3.75,  -2.5 ,  -1.25,
               0.  ,   1.25,   2.5 ,   3.75,   5.  ,   6.25,   7.5 ,   8.75,  10.  ,
              11.25,  12.5 ,  13.75,  15.  ,  16.25,  17.5 ,  18.75,  20.  ,  21.25,
              22.5 ,  23.75,  25.  ,  26.25,  27.5 ,  28.75,  30.  ,  31.25,  32.5 ,
              33.75,  35.  ,  36.25,  37.5 ,  38.75,  40.  ,  41.25,  42.5 ,  43.75,
              45.  ,  46.25,  47.5 ,  48.75,  50.  ,  51.25,  52.5 ,  53.75,  55.  ,
              56.25,  57.5 ,  58.75,  60.  ,  61.25,  62.5 ,  63.75,  65.  ,  66.25,
              67.5 ,  68.75,  70.  ,  71.25,  72.5 ,  73.75,  75.  ,  76.25,  77.5 ,
              78.75,  80.  ,  81.25,  82.5 ,  83.75,  85.  ,  86.25,  87.5 ,  88.75,
              90.  ], dtype=float32)
    • lon
      (lon)
      float32
      0.0 1.875 3.75 ... 356.25 358.125
      units :
      degrees_east
      standard_name :
      longitude
      modulo :
      360.0
      point_spacing :
      even
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125],
            dtype=float32)
    • time
      (time)
      datetime64[ns]
      1978-01-16T12:00:00 ... 2011-12-16T12:00:00
      standard_name :
      time
      axis :
      T
      array(['1978-01-16T12:00:00.000000000', '1978-02-15T00:00:00.000000000',
             '1978-03-16T12:00:00.000000000', ..., '2011-10-16T12:00:00.000000000',
             '2011-11-16T00:00:00.000000000', '2011-12-16T12:00:00.000000000'],
            dtype='datetime64[ns]')
  • stash_item :
    236
    stash_model :
    1
    lookup_source :
    defaults (cdunifpp V0.13)
    long_name :
    air temperature at 1.5 m
    cell_methods :
    time: mean
    units :
    K
    stash_section :
    3
    name :
    tas
    standard_name :
    air_temperature

Calculations using dimension names

Just like numpy, arrays have common functions as methods. But unlike numpy, you can identify dimensions by name instead of index position.

Xarray arrays work with most numpy functions. If not, you can access the underlying numpy array.

In [34]:
print('Global mean\n',tas.mean(),'\n')
print('Latitudinal mean\n',tas.mean(dim='lat'))
Global mean
 <xarray.DataArray 'tas' ()>
array(278.28284, dtype=float32) 

Latitudinal mean
 <xarray.DataArray 'tas' (time: 408, lon: 192)>
array([[278.467  , 278.4859 , 278.38632, ..., 278.32104, 278.27985,
        278.3821 ],
       [278.21014, 278.16296, 278.00296, ..., 278.27924, 278.2395 ,
        278.24582],
       [277.4103 , 277.37396, 277.2726 , ..., 277.37128, 277.35922,
        277.40472],
       ...,
       [278.6603 , 278.71646, 278.6278 , ..., 278.4839 , 278.49448,
        278.61685],
       [279.5895 , 279.64603, 279.59464, ..., 279.65546, 279.55872,
        279.58517],
       [279.07565, 279.1083 , 279.01367, ..., 279.26144, 279.12497,
        279.1093 ]], dtype=float32)
Coordinates:
  * lon      (lon) float32 0.0 1.875 3.75 5.625 ... 352.5 354.375 356.25 358.125
  * time     (time) datetime64[ns] 1978-01-16T12:00:00 ... 2011-12-16T12:00:00

Selection using coordinate values

Remember how the variables have the dimension names and the coordinate arrays attached to them? This means you can select data using the coordinate names and values rather than indexes.

In [35]:
tas.sel(lat=-85)
Out[35]:
Show/Hide data repr Show/Hide attributes
xarray.DataArray
'tas'
  • time: 408
  • lon: 192
  • 245.47575 245.20438 244.97054 ... 249.84563 249.65027 249.47154
    array([[245.47575, 245.20438, 244.97054, ..., 246.39372, 246.07375, 245.7676 ],
           [237.7508 , 237.50137, 237.2808 , ..., 238.60011, 238.29971, 238.01099],
           [223.22281, 222.64638, 222.1247 , ..., 225.09627, 224.4605 , 223.82877],
           ...,
           [225.40732, 225.0019 , 224.64072, ..., 226.89435, 226.37177, 225.8708 ],
           [243.86288, 243.60034, 243.36409, ..., 244.71707, 244.42044, 244.13028],
           [249.29846, 249.13568, 248.98572, ..., 249.84563, 249.65027, 249.47154]],
          dtype=float32)
    • lat
      ()
      float32
      -85.0
      units :
      degrees_north
      standard_name :
      latitude
      point_spacing :
      even
      axis :
      Y
      array(-85., dtype=float32)
    • lon
      (lon)
      float32
      0.0 1.875 3.75 ... 356.25 358.125
      units :
      degrees_east
      standard_name :
      longitude
      modulo :
      360.0
      point_spacing :
      even
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125],
            dtype=float32)
    • time
      (time)
      datetime64[ns]
      1978-01-16T12:00:00 ... 2011-12-16T12:00:00
      standard_name :
      time
      axis :
      T
      array(['1978-01-16T12:00:00.000000000', '1978-02-15T00:00:00.000000000',
             '1978-03-16T12:00:00.000000000', ..., '2011-10-16T12:00:00.000000000',
             '2011-11-16T00:00:00.000000000', '2011-12-16T12:00:00.000000000'],
            dtype='datetime64[ns]')
  • stash_item :
    236
    stash_model :
    1
    lookup_source :
    defaults (cdunifpp V0.13)
    long_name :
    air temperature at 1.5 m
    cell_methods :
    time: mean
    units :
    K
    stash_section :
    3
    name :
    tas
    standard_name :
    air_temperature

Xarray can even interpolate for you. You don't have to know the exact values of the points that are in your array. For example you could ask for the nearest points to 100°E in longitude. I would not use this to actually interpolate a whole field! Just use it to save on typing or if you have a projected grid.

In [36]:
tas.sel(lat=-87, method='nearest')
Out[36]:
Show/Hide data repr Show/Hide attributes
xarray.DataArray
'tas'
  • time: 408
  • lon: 192
  • 244.72847 244.6048 244.46696 ... 249.38219 249.31053 249.22963
    array([[244.72847, 244.6048 , 244.46696, ..., 245.10951, 244.98584, 244.85846],
           [237.15535, 237.08058, 237.00803, ..., 237.37787, 237.30826, 237.23053],
           [222.5073 , 222.38301, 222.26732, ..., 222.87436, 222.75264, 222.6297 ],
           ...,
           [225.26901, 225.17426, 225.08258, ..., 225.56303, 225.46518, 225.36763],
           [242.8312 , 242.72885, 242.62749, ..., 243.12952, 243.03493, 242.93588],
           [249.1559 , 249.08406, 249.00839, ..., 249.38219, 249.31053, 249.22963]],
          dtype=float32)
    • lat
      ()
      float32
      -87.5
      units :
      degrees_north
      standard_name :
      latitude
      point_spacing :
      even
      axis :
      Y
      array(-87.5, dtype=float32)
    • lon
      (lon)
      float32
      0.0 1.875 3.75 ... 356.25 358.125
      units :
      degrees_east
      standard_name :
      longitude
      modulo :
      360.0
      point_spacing :
      even
      axis :
      X
      array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
              15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
              30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
              45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
              60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
              75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
              90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
             105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
             120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
             135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
             150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
             165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
             180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
             195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
             210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
             225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
             240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
             255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
             270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
             285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
             300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
             315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
             330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
             345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125],
            dtype=float32)
    • time
      (time)
      datetime64[ns]
      1978-01-16T12:00:00 ... 2011-12-16T12:00:00
      standard_name :
      time
      axis :
      T
      array(['1978-01-16T12:00:00.000000000', '1978-02-15T00:00:00.000000000',
             '1978-03-16T12:00:00.000000000', ..., '2011-10-16T12:00:00.000000000',
             '2011-11-16T00:00:00.000000000', '2011-12-16T12:00:00.000000000'],
            dtype='datetime64[ns]')
  • stash_item :
    236
    stash_model :
    1
    lookup_source :
    defaults (cdunifpp V0.13)
    long_name :
    air temperature at 1.5 m
    cell_methods :
    time: mean
    units :
    K
    stash_section :
    3
    name :
    tas
    standard_name :
    air_temperature

Quick plotting

It is very easy to do a quick plot of your data. It isn't a plot ready for publication but it can easily allow you to visualise your fields. And the nice touch is Xarray will automatically use the meta-data to add labelling to the plot.

In [37]:
tas.sel(time='1978-01').plot()
Out[37]:
<matplotlib.collections.QuadMesh at 0x1280e6310>

Note we haven't loaded matplotlib, but because xarray uses it, it loads it for us.

Save data to file

There is a very simple way to save data back to a netcdf file. It isn't necessarily the fastest way. But you should only have to write out some analysed fields using Python which means relatively small amounts of data.

Note, netcdf support inline compression so you should ALWAYS save your data compressed. Inline compression means the file just looks the same, access is the same, you don't need to uncompress before being able to see the information from the file.

In [ ]:
# Compression
encod={}
for var in ds.data_vars:   # data_vars stores the names of the variables in a dataset, as strings
    encod[var]={'zlib':True}

# Write to file
ds.to_netcdf('test.nc',encoding=encod)

More info

This was a very very quick presentation of xarray. We were simply trying to give you the very basics, especially highlights the philosophy of the package. We will run a xarray training soon. In the meantime if you want to know more, you can always run through xarray quick overview

In addition, the CMS team has a blog with quite a few blogs using xarray and Python in general. Feel free to check those as well.

In [ ]: