.. _sig-menu-processing: Processing Signals ================== This section describes the signal processing features available in DataLab. .. seealso:: :ref:`sig-menu-operations` for more information on operations that can be performed on signals, or :ref:`sig-menu-analysis` for information on analysis features on signals. .. figure:: /images/shots/s_processing.png Screenshot of the "Processing" menu. When the "Signal Panel" is selected, the menus and toolbars are updated to provide signal-related actions. The "Processing" menu allows you to perform various processing on the selected signals, such as smoothing, normalization, or interpolation. Axis transformation ^^^^^^^^^^^^^^^^^^^ Linear calibration ~~~~~~~~~~~~~~~~~~ Create a new signal which is a linear calibration of each selected signal with respect to X or Y axis: .. list-table:: :header-rows: 1 :widths: 40, 60 * - Parameter - Linear calibration * - X-axis - :math:`x_{1} = a.x_{0} + b` * - Y-axis - :math:`y_{1} = a.y_{0} + b` Swap X/Y axes ~~~~~~~~~~~~~ Create a new signal which is the result of swapping X/Y data. Reverse X-axis ~~~~~~~~~~~~~~ Create a new signal which is the result of reversing X data. Replace X by other signal's Y ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Create a new signal by combining two signals, using Y values from the second signal as X coordinates for the first signal's Y values. This feature is particularly useful for calibration scenarios, such as: - Applying wavelength calibration to spectroscopy data - Using a calibrated reference signal to rescale measurement data - Combining signals where one contains calibration points The two signals must have the same number of points. The operation directly uses the Y arrays without interpolation. .. list-table:: :header-rows: 1 :widths: 40, 60 * - Input - Description * - Signal 1 - Provides the Y data for the output signal * - Signal 2 - Provides the Y data that becomes the X coordinates of the output signal X-Y mode ~~~~~~~~ Create a new signal by simulating the X-Y mode of an oscilloscope. This operation plots one signal as a function of another by: 1. Finding the overlapping X range between both signals 2. Resampling both signals onto a common X grid within that overlap 3. Using the resampled Y values as coordinates (first signal's Y as X, second signal's Y as Y) .. note:: This is different from "Replace X by other signal's Y" as it performs interpolation to handle signals with different X arrays. Convert to Cartesian coordinates ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Create a new signal which is the result of converting polar coordinates to Cartesian coordinates. This function assumes that the x-axis represents the radius and the y-axis the angle. .. warning:: A radius cannot be negative. Any negative value is clipped to 0. Convert to polar coordinates ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Create a new signal which is the result of converting Cartesian coordinates to polar coordinates. Level adjustment ^^^^^^^^^^^^^^^^ Normalize ~~~~~~~~~ Create a new signal which is the normalization of each selected signal by maximum, amplitude, sum, energy or RMS: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Parameter - Normalization * - Maximum - :math:`y_{1}= \dfrac{y_{0}}{\max\left(y_{0}\right)}` * - Amplitude - :math:`y_{1}= \dfrac{y_{0}'}{\max\left(y_{0}'\right)}` with :math:`y_{0}'=y_{0}-\min\left(y_{0}\right)` * - Area - :math:`y_{1}= \dfrac{y_{0}}{\sum_{n=0}^{N}y_{0}[n]}` * - Energy - :math:`y_{1}= \dfrac{y_{0}}{\sqrt{\sum_{n=0}^{N}\left|y_{0}[n]\right|^2}}` * - RMS - :math:`y_{1}= \dfrac{y_{0}}{\sqrt{\dfrac{1}{N}\sum_{n=0}^{N}\left|y_{0}[n]\right|^2}}` Clipping ~~~~~~~~ Create a new signal which is the result of clipping each selected signal. Offset correction ~~~~~~~~~~~~~~~~~ Create a new signal which is the result of offset correction of each selected signal. This operation is performed by subtracting the signal baseline which is estimated by the mean value of a user-defined range. Noise addition ^^^^^^^^^^^^^^ Generate new signals by adding the same noise to each selected signal. The available noise types are: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Noise - Description * - Gaussian - Normal distribution * - Uniform - Uniform distribution * - Poisson - Poisson distribution Noise reduction ^^^^^^^^^^^^^^^ Create a new signal which is the result of noise reduction of each selected signal. The following filters are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Filter - Formula/implementation * - Gaussian filter - `scipy.ndimage.gaussian_filter `_ * - Moving average - `scipy.ndimage.uniform_filter `_ * - Moving median - `scipy.ndimage.median_filter `_ * - Wiener filter - `scipy.signal.wiener `_ Fourier analysis ^^^^^^^^^^^^^^^^ Zero padding ~~~~~~~~~~~~ Create a new signal which is the result of zero padding of each selected signal. The following parameters are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Parameter - Description * - Strategy - Zero padding strategy (see below) * - Number of points - Custom length (if `strategy` is "custom") Zero padding strategy refers to the method used to increase the length of the signal, and it can be one of the following: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Strategy - Description * - next_pow2 - Next power of 2 (e.g. 1024 → 2048) * - double - Double the length (e.g. 1024 → 2048) * - triple - Triple the length (e.g. 1024 → 3072) * - custom - Custom length (user-defined) FFT related functions ~~~~~~~~~~~~~~~~~~~~~ Create a new signal which is the result of a Fourier analysis of each selected signal. The following functions are available: .. list-table:: :header-rows: 1 :widths: 20, 30, 50 * - Function - Description - Formula/implementation * - FFT - Fast Fourier Transform - `numpy.fft.fft `_ * - Inverse FFT - Inverse Fast Fourier Transform - `numpy.fft.ifft `_ * - Magnitude spectrum - Optional: output in decibels (dB) - | :math:`y_{1} = \left|\FFT\left(y_{0}\right)\right|` or | :math:`y_{1} = 20 \log_{10} \left(\left|\FFT\left(y_{0}\right)\right|\right)` (dB) * - Phase spectrum - Phase of the FFT expressed in degrees, using `numpy.angle `_ - :math:`y_{1} = \angle \FFT\left(y_{0}\right)` * - Power spectral density (PSD) - Optional: output in decibels (dB). PSD is estimated using Welch's method (see `scipy.signal.welch `_) - | :math:`y_{1} = \PSD\left(y_{0}\right)` or | :math:`y_{1} = 10 \log_{10} \left(\PSD\left(y_{0}\right)\right)` (dB) .. note:: FFT and inverse FFT are performed using frequency shifting if the option is enabled in DataLab settings (see :ref:`settings`). Frequency filters ^^^^^^^^^^^^^^^^^ Create a new signal which is the result of applying a frequency filter to each selected signal. The following filters are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Filter - Description * - |lowpass| Low-pass - Filter out high frequencies, above a cutoff frequency * - |highpass| High-pass - Filter out low frequencies, below a cutoff frequency * - |bandpass| Band-pass - Filter out frequencies outside a range * - |bandstop| Band-stop - Filter out frequencies inside a range .. |lowpass| image:: ../../../datalab/data/icons/processing/lowpass.svg :width: 24px :height: 24px :class: dark-light no-scaled-link .. |highpass| image:: ../../../datalab/data/icons/processing/highpass.svg :width: 24px :height: 24px :class: dark-light no-scaled-link .. |bandpass| image:: ../../../datalab/data/icons/processing/bandpass.svg :width: 24px :height: 24px :class: dark-light no-scaled-link .. |bandstop| image:: ../../../datalab/data/icons/processing/bandstop.svg :width: 24px :height: 24px :class: dark-light no-scaled-link For each filter, the following methods are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Method - Description * - Bessel - Bessel filter, using SciPy's `scipy.signal.bessel `_ function * - Brick wall - Ideal (rectangular) filter in the frequency domain * - Butterworth - Butterworth filter, using SciPy's `scipy.signal.butter `_ function * - Chebyshev I - Chebyshev type I filter, using SciPy's `scipy.signal.cheby1 `_ function * - Chebyshev II - Chebyshev type II filter, using SciPy's `scipy.signal.cheby2 `_ function * - Elliptic - Elliptic filter, using SciPy's `scipy.signal.ellip `_ function Fitting ^^^^^^^ Fit a model to each selected signal. This can be done automatically or through an interactive curve fitting dialog. .. list-table:: :header-rows: 1 :widths: 20, 80 * - Model - Equation * - Linear - :math:`y = c_{0} + c_{1} \cdot x` * - Polynomial - :math:`y = c_{0} + c_{1} \cdot x + c_{2} \cdot x^2 + ... + c_{n} \cdot x^n` * - Gaussian - :math:`y = y_{0} + \dfrac{A}{\sqrt{2 \pi} \sigma} \exp\left(-\dfrac{1}{2} \left(\dfrac{x - x_{0}}{\sigma}\right)^2\right)` * - Lorentzian - :math:`y = y_{0} + \dfrac{A}{\pi \sigma} \cdot \dfrac{1}{1+\left(\dfrac{x-x_{0}}{\sigma}\right)^2}` * - Voigt - :math:`y = y_{0} + A \cdot \dfrac{\Re\left(\exp(-z^2) \cdot \operatorname{erfc}(-j \cdot z)\right)}{\sqrt{2 \pi} \sigma}` with :math:`z = \dfrac{x - x_{0} - j \cdot \sigma}{\sqrt{2} \sigma}` * - Multi-Gaussian - :math:`y = y_{0} + \sum_{i=0}^{N}\dfrac{A_{i}}{\sqrt{2 \pi} \sigma_{i}} \exp\left(-\dfrac{1}{2} \left(\dfrac{x - x_{0,i}}{\sigma_{i}}\right)^2\right)` * - Multi-Lorentzian - :math:`y = y_{0} + \sum_{i=0}^{N}\dfrac{A_{i}}{\pi \sigma_{i}} \cdot \dfrac{1}{1 + \left(\dfrac{x - x_{0,i}}{\sigma_{i}}\right)^2}` * - Planck - :math:`y = y_{0} + A\cdot \dfrac{2h \cdot c^2}{\lambda^5} \cdot \left(\exp\left(\dfrac{h \cdot c}{\lambda \cdot k \cdot T}\right)-1\right)^{-1}` * - Two half Gaussians - | :math:`y = y_{0} + A \cdot \dfrac{1}{\sqrt{2 \pi} \sigma_{0}} \cdot \exp\left(-\dfrac{1}{2}\left(\dfrac{x - x_{0}}{\sigma_{0}}\right)^2\right)` if :math:`x < x_{0}` | :math:`y = y_{0} + A \cdot \dfrac{1}{\sqrt{2 \pi} \sigma_{1}} \cdot \exp\left(-\dfrac{1}{2}\left(\dfrac{x - x_{1}}{\sigma_{1}}\right)^2\right)` otherwise * - Two back-to-back exponentials - | :math:`y = y_{0} + A_{1} \cdot \exp\left(-x/\tau_{1}\right)` if :math:`x < 0` | :math:`y = y_{0} + A_{2} \cdot \exp(-x/\tau_{2})` otherwise * - Exponential - :math:`y = y_{0} + A \exp\left(B \cdot x\right)` * - Sinusoidal - :math:`y = y_{0} + A \sin\left(2 \pi f \cdot x + \phi\right)` * - Cumulative Distribution Function (CDF) - :math:`y = y_{0} + A \erf\left(\dfrac{x - x_{0}}{\sqrt{2} \sigma}\right)` Windowing ^^^^^^^^^ Create a new signal which is the result of applying a window function to each selected signal. The following window functions are available: .. list-table:: :header-rows: 1 :widths: 20, 80 * - Window function - Reference * - Barthann - :py:func:`scipy.signal.windows.barthann` * - Bartlett - :py:func:`numpy.bartlett` * - Blackman - :py:func:`scipy.signal.windows.blackman` * - Blackman-Harris - :py:func:`scipy.signal.windows.blackmanharris` * - Bohman - :py:func:`scipy.signal.windows.bohman` * - Boxcar (rectangular) - :py:func:`scipy.signal.windows.boxcar` * - Cosine - :py:func:`scipy.signal.windows.cosine` * - Exponential - :py:func:`scipy.signal.windows.exponential` * - Flat top - :py:func:`scipy.signal.windows.flattop` * - Gaussian - :py:func:`scipy.signal.windows.gaussian` * - Hamming - :py:func:`numpy.hamming` * - Hann - :py:func:`numpy.hanning` * - Kaiser - :py:func:`scipy.signal.windows.kaiser` * - Lanczos - :py:func:`scipy.signal.windows.lanczos` * - Nuttall - :py:func:`scipy.signal.windows.nuttall` * - Parzen - :py:func:`scipy.signal.windows.parzen` * - Taylor - :py:func:`scipy.signal.windows.taylor` * - Tukey - :py:func:`scipy.signal.windows.tukey` Detrending ^^^^^^^^^^ Create a new signal which is the detrending of each selected signal. This features is based on SciPy's `scipy.signal.detrend `_ function. The following parameters are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Parameter - Description * - Method - Detrending method: 'linear' or 'constant'. See SciPy's `scipy.signal.detrend `_ function. Interpolation ^^^^^^^^^^^^^ Create a new signal which is the interpolation of each selected signal with respect to a second signal X-axis (which might be the same as one of the selected signals). The following interpolation methods are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Method - Description * - Linear - Linear interpolation, using using NumPy's `interp `_ function * - Spline - Cubic spline interpolation, using using SciPy's `scipy.interpolate.splev `_ function * - Quadratic - Quadratic interpolation, using using NumPy's `polyval `_ function * - Cubic - Cubic interpolation, using using SciPy's `Akima1DInterpolator `_ class * - Barycentric - Barycentric interpolation, using using SciPy's `BarycentricInterpolator `_ class * - PCHIP - Piecewise Cubic Hermite Interpolating Polynomial (PCHIP) interpolation, using using SciPy's `PchipInterpolator `_ class Resampling ^^^^^^^^^^ Create a new signal which is the resampling of each selected signal. The following parameters are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Parameter - Description * - Method - Interpolation method (see previous section) * - Fill value - Interpolation fill value (see previous section) * - Xmin - Minimum X value * - Xmax - Maximum X value * - Mode - Resampling mode: step size or number of points * - Step size - Resampling step size * - Number of points - Resampling number of points Stability analysis ^^^^^^^^^^^^^^^^^^ Create a new signal which is the result of a stability analysis of each selected signal. The following stability analysis methods are available: .. list-table:: :header-rows: 1 :widths: 25, 75 * - Function - Description * - Allan variance - Measure of the stability of a signal: defined as the variance of the difference between two successive measurements as a function of the time interval between them. * - Allan deviation - Square root of the Allan variance. * - Overlapping Allan deviation - A more robust version of the Allan variance that overlaps successive segments to improve statistical confidence. * - Modified Allan variance - A variation of the Allan variance that accounts for phase noise by introducing a filtering operation. * - Hadamard variance - An alternative to Allan variance, more robust to linear frequency drift in the signal * - Total variance - Extends the Allan variance concept to cover all possible averaging intervals. * - Time deviation - Derived from Allan deviation, quantifies stability in terms of time rather than frequency. .. note:: The "All stability features" option allows to compute all stability analysis methods at once.