FAQ

This document contains frequently asked questions (FAQs) about the cdasws Python library.

  1. Does the cdasws library support HTTP persistent connections? Yes. The cdasws library uses the python requests library including its support of persistent connections and urllib3's connection pooling.

  2. Does the cdasws client library support caching and HTTP conditional GETs? No. Not at this time but this may be added in the future since the CDAS web services support this.

  3. Does the cdasws library support the existing CDAS web service requests that produce higher-level products such as plots, movies/animations, CSV, netCDF, audio, and CDFML? Yes.

  4. How does this library differ from the Heliophysics Application Programming Interface (HAPI) client python library and the ai.cdas python library? The CDAS web services and the HAPI service are fundamentally different services which can provide functionally similar access to the CDAS data. The following table summarizes the different features of the libraries for accessing data from NASA's Coordinated Data Analysis System repository.

Feature Support cdasws ai.cdas hapi
SpacePy datamodel yes no no
xarray.Dataset yes no no
pandas.DataFrame yes no no
Includes all source6 and ISTP/SPDF metadata yes no no
Streaming data reception no no no1
Binary data transfer yes yes yes
Compressed data transfer yes yes no5
Access to cdaweb netCDF data sets yes yes no4
Access to non-cdaweb data sets no no yes
Search server datasets (e.g., by instrument type) yes yes no
Data gap information yes no no
Data binning yes no no
CDAWeb plotting, movies, audification, etc. yes no no
Dependent upon CDF (C) library no2 no3 no
Access by Digital Object Identifiers(DOIs) yes ? no
Access by SPASE Resource Identifiers yes ? no

  1. Although the HAPI protocol was specifically designed for data streaming, at this time, the HAPI python client library does not support streaming data reception. That is, the library waits until all data has been received before returning any of it to the caller. ↩

  2. A client that does not call the get_data method or one that wants the data returned in an xarray.Dataset is not dependent upon the CDF C library (or SpacePy). However, to return data in the SpacePy data model, the get_data method is dependent on pycdf which is dependent upon the cdf C library. ↩

  3. ai.cdas can be used with or without CDF. CDF can be avoided by transfering the data in an ASCII representation. ↩

  4. At this time, cdaweb's HAPI server does not support access to datasets with netCDF files (for example, FORMOSAT5_AIP_IDN, GOLD_L2_NMAX, GOLD_L2_ON2, GOLD_L2_TDISK, ICON_L2-7_IVM-A, PMC-TURBO_L1_BOLIDE_VBC). This is a current limitation of the cdaweb HAPI server and not of the HAPI protocol. ↩

  5. HAPI is a streaming protocol which for HTTP 1.1, requires chunked transfer encoding. When chunked transfer encoding is combined with HTTP compression, the content stream is first compressed, then chunked; so the chunk encoding itself is not compressed, and the data in each chunk is not compressed individually. The client then decodes the stream by concatenating the chunks and uncompressing the result. So the use of HTTP compression defeats the purpose of the streaming. Although compression is technically possible. ↩

  6. All source metadata includes project specific metadata. For example, the metadata below shows a few additional values for the MMS1_EDP_FAST_L2_DCE dataset that is only available through the cdasw API: DATA_VERSION: v3.0.0 CALIBRATION_FILE: /mms/itfhome/mms-fieldspro/cal/mms1_edp_fast... GENERATION_DATE: 20230308 GENERATED_BY: IRFU Matlab v1.10.0 and LASP IDL v8.4 while ... SKELETON_VERSION: v0.0.8 PARENTS: ['CDF>mms1_edp_fast_l2pre_dce_20230204_v3.0.... CDAWEB_PARENTS: ['mms1_edp_fast_l2_dce_00000000_v01', 'mms1_... ... ↩