.. _drms-tutorial: ******** Tutorial ******** This tutorial gives an introduction on how to use the ``drms`` Python library. More detailed information on the different classes and functions can be found in the :ref:`API Reference `. Basic usage =========== We start with looking at data series that are available from `JSOC `__ and perform some basic DRMS queries to obtain keyword data (metadata) and segment file (data) locations. This is essentially what you can do on the `JSOC Lookdata `__ website. To be able to access the JSOC DRMS from Python, we first need to import the ``drms`` module and create an instance of the `~drms.client.Client` class: .. code-block:: python >>> import drms >>> client = drms.Client() # doctest: +REMOTE_DATA All available data series can be now retrieved by calling :meth:`drms.client.Client.series`. HMI series names start with ``"hmi."``, AIA series names with ``"aia."`` and the names of MDI series with ``"mdi."``. The first (optional) parameter of this method takes a regular expression that allows you to filter the result. If for example, you want to obtain a list of HMI series, with a name that start with the string ``"m_"``, you can write: .. code-block:: python >>> client.series(r'hmi\.m_') # doctest: +REMOTE_DATA ['hmi.M_45s', 'hmi.M_45s_dcon', 'hmi.M_720s', 'hmi.M_720s_dcon', 'hmi.M_720s_dconS', 'hmi.m_720s_mod', 'hmi.m_720s_nrt'] Keep in mind to escape the dot character (``'.'``), like it is shown in the example above, if you want to include it in your filter string. Also note that series names are handled in a case-insensitive way. DRMS records can be selected by creating a query string that contains a series name, followed by one or more fields, which are surrounded by square brackets. Each of those fields corresponds to a specific primekey, that is specified in the series definition. A complete set of primekeys represents a unique identifier for a record in that particular series. For more detailed information on building record set queries, including additional non-primekey fields, see the `JSOC Help `__ page about this. With the ``drms`` module you can use :meth:`drms.client.Client.pkeys` to obtain a list of all primekeys of a series: .. code-block:: python >>> client.pkeys('hmi.m_720s') # doctest: +REMOTE_DATA ['T_REC', 'CAMERA'] >>> client.pkeys('hmi.v_sht_modes') # doctest: +REMOTE_DATA ['T_START', 'LMIN', 'LMAX', 'NDT'] A list of all (regular) keywords can be obtained using :meth:`drms.client.Client.keys`. You can also use :meth:`drms.client.Client.info` to get more detailed information about a series: .. code-block:: python >>> series_info = client.info('hmi.v_sht_2drls') # doctest: +REMOTE_DATA >>> series_info.segments # doctest: +REMOTE_DATA type units protocol dims note name split string none generic calculated splittings rot string none generic rotation profile err string none generic errors mesh string none generic radial grid points parms string none generic input parameters log string none generic standard output All table-like structures, returned by routines in the ``drms`` module, are `Pandas DataFrames `__. If you are new to `Pandas `__, you should have a look at the introduction to `Pandas Data Structures `__. Record set queries, used to obtain keyword data and get the location of data segments, can be performed using :meth:`drms.client.Client.query`. To get, for example, the record time and the mean value for some of the HMI Dopplergrams that were recorded on April 1, 2016, together with the spacecraft's radial velocity in respect to the Sun, you can write: .. code-block:: python >>> query = client.query('hmi.v_45s[2016.04.01_TAI/1d@6h]', ... key='T_REC, DATAMEAN, OBS_VR') # doctest: +REMOTE_DATA >>> query # doctest: +REMOTE_DATA T_REC DATAMEAN OBS_VR 0 2016.04.01_00:00:00_TAI 3313.104980 3309.268006 1 2016.04.01_06:00:00_TAI 878.075195 887.864139 2 2016.04.01_12:00:00_TAI -2289.062500 -2284.690263 3 2016.04.01_18:00:00_TAI 128.609283 137.836168 JSOC time strings can be converted to a naive `~datetime.datetime` representation using :meth:`drms.utils.to_datetime`: .. code-block:: python >>> timestamps = drms.to_datetime(query.T_REC) # doctest: +REMOTE_DATA >>> timestamps # doctest: +REMOTE_DATA 0 2016-04-01 00:00:00 1 2016-04-01 06:00:00 2 2016-04-01 12:00:00 3 2016-04-01 18:00:00 Name: T_REC, dtype: datetime64[us] For most of the HMI and MDI data sets, the `TAI `__ time standard is used which, in contrast to `UTC `__, does not make use of any leap seconds. The TAI standard is currently not supported by the Python standard libraries. If you need to convert timestamps between TAI and UTC, you can use `Astropy `__: .. code-block:: python >>> from astropy.time import Time >>> start_time = Time(timestamps[0], format='datetime', scale='tai') # doctest: +REMOTE_DATA >>> start_time # doctest: +REMOTE_DATA