cdasws Binning Example Jupyter Notebook

This Jupyter notebook demonstrates using the cdasws Python package to retrieve data from two cdaweb datasets that have different timestamps and places the data on the same time grid (with optional spike removal). The data is returned in the SpacePy data model. The AC_H0_SWE and AC_H2_SWE datasets are used in this example. For more information on binning, see binning in cdaweb.

This notebook contains the following sections:

1. Prerequisites
2. Setup
3. Display Original Data
4. Get Binned Data
5. Compare Data
6. Additional Documentation

Prerequisites

Install the prerequisite software the Python Package Index.

1. pip install spacepy
2. pip install cdasws

Setup

Execute some preliminary code that is necessary before the code that follows.

from cdasws import CdasWs

import matplotlib.pyplot as plt
    
cdas = CdasWs()

Display Original Data

The following code gets and displays the original, unbinned data.

dataset0 = 'AC_H0_SWE'
parameters = ['Np']
start = '1998-02-04T00:00:00Z'
stop = '1998-02-06T00:00:00Z'
status, data0 = cdas.get_data(dataset0, parameters, start, stop)
print(f'{dataset0}')
print(data0)
dataset1 = 'AC_H2_SWE'
status, data1 = cdas.get_data(dataset1, parameters, start, stop)
print()
print(f'{dataset1}')
print(data1)

AC_H0_SWE
{'Epoch': VarCopy([datetime.datetime(1998, 2, 4, 0, 0, 31),
         datetime.datetime(1998, 2, 4, 0, 1, 35),
         datetime.datetime(1998, 2, 4, 0, 2, 39), ...,
         datetime.datetime(1998, 2, 5, 23, 57, 19),
         datetime.datetime(1998, 2, 5, 23, 58, 23),
         datetime.datetime(1998, 2, 5, 23, 59, 27)], dtype=object), 'Np': VarCopy([-1.00000e+31, -1.00000e+31, -1.00000e+31, ...,  1.44819e+01,
          1.49483e+01,  1.35783e+01], dtype=float32)}

AC_H2_SWE
{'Epoch': VarCopy([datetime.datetime(1998, 2, 4, 0, 0),
         datetime.datetime(1998, 2, 4, 1, 0),
         datetime.datetime(1998, 2, 4, 2, 0),
         datetime.datetime(1998, 2, 4, 3, 0),
         datetime.datetime(1998, 2, 4, 4, 0),
         datetime.datetime(1998, 2, 4, 5, 0),
         datetime.datetime(1998, 2, 4, 6, 0),
         datetime.datetime(1998, 2, 4, 7, 0),
         datetime.datetime(1998, 2, 4, 8, 0),
         datetime.datetime(1998, 2, 4, 9, 0),
         datetime.datetime(1998, 2, 4, 10, 0),
         datetime.datetime(1998, 2, 4, 11, 0),
         datetime.datetime(1998, 2, 4, 12, 0),
         datetime.datetime(1998, 2, 4, 13, 0),
         datetime.datetime(1998, 2, 4, 14, 0),
         datetime.datetime(1998, 2, 4, 15, 0),
         datetime.datetime(1998, 2, 4, 16, 0),
         datetime.datetime(1998, 2, 4, 17, 0),
         datetime.datetime(1998, 2, 4, 18, 0),
         datetime.datetime(1998, 2, 4, 19, 0),
         datetime.datetime(1998, 2, 4, 20, 0),
         datetime.datetime(1998, 2, 4, 21, 0),
         datetime.datetime(1998, 2, 4, 22, 0),
         datetime.datetime(1998, 2, 4, 23, 0),
         datetime.datetime(1998, 2, 5, 0, 0),
         datetime.datetime(1998, 2, 5, 1, 0),
         datetime.datetime(1998, 2, 5, 2, 0),
         datetime.datetime(1998, 2, 5, 3, 0),
         datetime.datetime(1998, 2, 5, 4, 0),
         datetime.datetime(1998, 2, 5, 5, 0),
         datetime.datetime(1998, 2, 5, 6, 0),
         datetime.datetime(1998, 2, 5, 7, 0),
         datetime.datetime(1998, 2, 5, 8, 0),
         datetime.datetime(1998, 2, 5, 9, 0),
         datetime.datetime(1998, 2, 5, 10, 0),
         datetime.datetime(1998, 2, 5, 11, 0),
         datetime.datetime(1998, 2, 5, 12, 0),
         datetime.datetime(1998, 2, 5, 13, 0),
         datetime.datetime(1998, 2, 5, 14, 0),
         datetime.datetime(1998, 2, 5, 15, 0),
         datetime.datetime(1998, 2, 5, 16, 0),
         datetime.datetime(1998, 2, 5, 17, 0),
         datetime.datetime(1998, 2, 5, 18, 0),
         datetime.datetime(1998, 2, 5, 19, 0),
         datetime.datetime(1998, 2, 5, 20, 0),
         datetime.datetime(1998, 2, 5, 21, 0),
         datetime.datetime(1998, 2, 5, 22, 0),
         datetime.datetime(1998, 2, 5, 23, 0),
         datetime.datetime(1998, 2, 6, 0, 0)], dtype=object), 'Np': VarCopy([-1.00000e+31, -1.00000e+31, -1.00000e+31, -1.00000e+31,
         -1.00000e+31, -1.00000e+31, -1.00000e+31, -1.00000e+31,
         -1.00000e+31, -1.00000e+31, -1.00000e+31, -1.00000e+31,
         -1.00000e+31, -1.00000e+31, -1.00000e+31, -1.00000e+31,
         -1.00000e+31, -1.00000e+31, -1.00000e+31, -1.00000e+31,
         -1.00000e+31, -1.00000e+31, -1.00000e+31,  1.50185e+01,
          1.66551e+01,  2.06010e+01,  1.86789e+01,  1.84808e+01,
          2.83151e+01,  2.29591e+01,  2.67127e+01,  2.87503e+01,
          3.11556e+01,  2.98451e+01,  2.96215e+01,  3.19601e+01,
          2.88363e+01,  2.34173e+01,  2.58818e+01,  2.50207e+01,
          2.76192e+01,  2.56742e+01,  2.56151e+01,  2.00080e+01,
          1.49786e+01,  1.20008e+01,  1.34287e+01,  1.04862e+01,
          1.06203e+01], dtype=float32)}

Get Binned Data

The following code gets data after it has been binned with 60 second time intervals and any missing values created by interpolation.

binData = {
    'interval': 60.0,
    'interpolateMissingValues': True,
    'sigmaMultiplier': 4
}
status, data0 = cdas.get_data(dataset0, parameters, start, stop, binData=binData)
print(f'{dataset0}')
print(data0)
status, data1 = cdas.get_data(dataset1, parameters, start, stop, binData=binData)
print()
print(f'{dataset1}')
print(data1)

AC_H0_SWE
{'Epoch_bin': VarCopy([datetime.datetime(1998, 2, 4, 0, 0, 30),
         datetime.datetime(1998, 2, 4, 0, 1, 30),
         datetime.datetime(1998, 2, 4, 0, 2, 30), ...,
         datetime.datetime(1998, 2, 5, 23, 57, 30),
         datetime.datetime(1998, 2, 5, 23, 58, 30),
         datetime.datetime(1998, 2, 5, 23, 59, 30)], dtype=object), 'Epoch': VarCopy([datetime.datetime(1998, 2, 4, 0, 0, 31),
         datetime.datetime(1998, 2, 4, 0, 1, 35),
         datetime.datetime(1998, 2, 4, 0, 2, 39), ...,
         datetime.datetime(1998, 2, 5, 23, 57, 19),
         datetime.datetime(1998, 2, 5, 23, 58, 23),
         datetime.datetime(1998, 2, 5, 23, 59, 27)], dtype=object), 'Np': VarCopy([16.3343, 16.3343, 16.3343, ..., 14.4819, 14.9483, 13.5783],
        dtype=float32), 'NP_NBIN': VarCopy([-0., -0., -0., ...,  1.,  1.,  1.], dtype=float32), 'NP_BIN_DELTA_MINUS_VAR': VarCopy([-1.e+31, -1.e+31, -1.e+31, ..., -1.e+31, -1.e+31, -1.e+31],
        dtype=float32), 'NP_BIN_DELTA_PLUS_VAR': VarCopy([-1.e+31, -1.e+31, -1.e+31, ..., -1.e+31, -1.e+31, -1.e+31],
        dtype=float32)}

AC_H2_SWE
{'Epoch_bin': VarCopy([datetime.datetime(1998, 2, 4, 0, 0, 30),
         datetime.datetime(1998, 2, 4, 0, 1, 30),
         datetime.datetime(1998, 2, 4, 0, 2, 30), ...,
         datetime.datetime(1998, 2, 5, 23, 57, 30),
         datetime.datetime(1998, 2, 5, 23, 58, 30),
         datetime.datetime(1998, 2, 5, 23, 59, 30)], dtype=object), 'Epoch': VarCopy([datetime.datetime(1998, 2, 4, 0, 0),
         datetime.datetime(1998, 2, 4, 1, 0),
         datetime.datetime(1998, 2, 4, 2, 0),
         datetime.datetime(1998, 2, 4, 3, 0),
         datetime.datetime(1998, 2, 4, 4, 0),
         datetime.datetime(1998, 2, 4, 5, 0),
         datetime.datetime(1998, 2, 4, 6, 0),
         datetime.datetime(1998, 2, 4, 7, 0),
         datetime.datetime(1998, 2, 4, 8, 0),
         datetime.datetime(1998, 2, 4, 9, 0),
         datetime.datetime(1998, 2, 4, 10, 0),
         datetime.datetime(1998, 2, 4, 11, 0),
         datetime.datetime(1998, 2, 4, 12, 0),
         datetime.datetime(1998, 2, 4, 13, 0),
         datetime.datetime(1998, 2, 4, 14, 0),
         datetime.datetime(1998, 2, 4, 15, 0),
         datetime.datetime(1998, 2, 4, 16, 0),
         datetime.datetime(1998, 2, 4, 17, 0),
         datetime.datetime(1998, 2, 4, 18, 0),
         datetime.datetime(1998, 2, 4, 19, 0),
         datetime.datetime(1998, 2, 4, 20, 0),
         datetime.datetime(1998, 2, 4, 21, 0),
         datetime.datetime(1998, 2, 4, 22, 0),
         datetime.datetime(1998, 2, 4, 23, 0),
         datetime.datetime(1998, 2, 5, 0, 0),
         datetime.datetime(1998, 2, 5, 1, 0),
         datetime.datetime(1998, 2, 5, 2, 0),
         datetime.datetime(1998, 2, 5, 3, 0),
         datetime.datetime(1998, 2, 5, 4, 0),
         datetime.datetime(1998, 2, 5, 5, 0),
         datetime.datetime(1998, 2, 5, 6, 0),
         datetime.datetime(1998, 2, 5, 7, 0),
         datetime.datetime(1998, 2, 5, 8, 0),
         datetime.datetime(1998, 2, 5, 9, 0),
         datetime.datetime(1998, 2, 5, 10, 0),
         datetime.datetime(1998, 2, 5, 11, 0),
         datetime.datetime(1998, 2, 5, 12, 0),
         datetime.datetime(1998, 2, 5, 13, 0),
         datetime.datetime(1998, 2, 5, 14, 0),
         datetime.datetime(1998, 2, 5, 15, 0),
         datetime.datetime(1998, 2, 5, 16, 0),
         datetime.datetime(1998, 2, 5, 17, 0),
         datetime.datetime(1998, 2, 5, 18, 0),
         datetime.datetime(1998, 2, 5, 19, 0),
         datetime.datetime(1998, 2, 5, 20, 0),
         datetime.datetime(1998, 2, 5, 21, 0),
         datetime.datetime(1998, 2, 5, 22, 0),
         datetime.datetime(1998, 2, 5, 23, 0),
         datetime.datetime(1998, 2, 6, 0, 0)], dtype=object), 'Np': VarCopy([16.6551, 16.6551, 16.6551, ..., 10.4862, 10.4862, 10.4862],
        dtype=float32), 'NP_NBIN': VarCopy([-0., -0., -0., ..., -0., -0., -0.], dtype=float32), 'NP_BIN_DELTA_MINUS_VAR': VarCopy([-1.e+31, -1.e+31, -1.e+31, ..., -1.e+31, -1.e+31, -1.e+31],
        dtype=float32), 'NP_BIN_DELTA_PLUS_VAR': VarCopy([-1.e+31, -1.e+31, -1.e+31, ..., -1.e+31, -1.e+31, -1.e+31],
        dtype=float32)}

Compare Data

The following code compares the binned data from the two datasets by plotting the values.

plt.plot(data0['Epoch_bin'], data0['Np'])
plt.plot(data1['Epoch_bin'], data1['Np'])
plt.xlabel(data0['Epoch'].attrs['LABLAXIS'])
plt.xticks(rotation=45, ha='right')
plt.ylabel(data0['Np'].attrs['LABLAXIS'] + ' ' +
           data0['Np'].attrs['UNITS'])
plt.legend([dataset0, dataset1])
plt.show()

Additional Documentation

View the cdasws API for additional functions. Additional notebook examples are also available.