| 1 |
.. _scientific_validation: |
| 2 |
|
| 3 |
Technical Validation |
| 4 |
===================== |
| 5 |
|
| 6 |
.. meta:: |
| 7 |
:description: Validation in DataLab, the open-source scientific data analysis and visualization platform |
| 8 |
:keywords: DataLab, scientific, data, analysis, validation, ground-truth, analytical |
| 9 |
|
| 10 |
DataLab technical validation is based on two key concepts: |
| 11 |
**ground-truth data** and **analytical validation**. |
| 12 |
|
| 13 |
Ground-truth data |
| 14 |
----------------- |
| 15 |
|
| 16 |
Ground-truth data is data that is known to be correct. It is used to validate the |
| 17 |
results of computational analysis. |
| 18 |
|
| 19 |
In DataLab, ground-truth data may be obtained from a variety of sources, including: |
| 20 |
|
| 21 |
- Experimental data |
| 22 |
- Simulated data |
| 23 |
- Synthetic data |
| 24 |
- Data from a trusted source |
| 25 |
|
| 26 |
Analytical validation |
| 27 |
--------------------- |
| 28 |
|
| 29 |
Analytical validation is the process of comparing the results of computational analysis |
| 30 |
to ground-truth data. This is done to ensure that the results are accurate and reliable. |
| 31 |
|
| 32 |
In DataLab, analytical validation is implemented using a variety of techniques, including: |
| 33 |
|
| 34 |
- Cross-validation with an analytical model (from a trusted source, |
| 35 |
e.g. `SciPy <https://www.scipy.org/>`_ or `NumPy <https://numpy.org/>`_) |
| 36 |
- Statistical analysis |
| 37 |
- Visual inspection |
| 38 |
- Expert review |
| 39 |
|
| 40 |
Scope |
| 41 |
----- |
| 42 |
|
| 43 |
The scope of technical validation in DataLab includes all compute functions that |
| 44 |
operate on DataLab's signal and image objects (i.e. :class:`sigima.objects.SignalObj` and |
| 45 |
:class:`sigima.objects.ImageObj`). |
| 46 |
|
| 47 |
This includes functions for (all functions are named ``compute_<function_name>``): |
| 48 |
|
| 49 |
- Signal processing (:mod:`sigima.proc.signal`) |
| 50 |
- Image processing (:mod:`sigima.proc.image`) |
| 51 |
|
| 52 |
Implementation |
| 53 |
-------------- |
| 54 |
|
| 55 |
The tests are implemented using the `pytest <https://docs.pytest.org/en/latest/>`_ |
| 56 |
framework. |
| 57 |
|
| 58 |
When writing a new technical validation test, the following rules should be followed |
| 59 |
regarding the test function: |
| 60 |
|
| 61 |
- The test function should be named: |
| 62 |
|
| 63 |
- ``test_signal_<function_name>`` for signal compute functions |
| 64 |
- ``test_image_<function_name>`` for image compute functions |
| 65 |
|
| 66 |
.. note:: |
| 67 |
|
| 68 |
The ``signal`` or ``image`` prefix is used to indicate the type of object that the |
| 69 |
function operates on. It may be omitted if the function operates exclusively on |
| 70 |
one type of object (e.g. ``test_adjust_gamma`` is the test function for the |
| 71 |
``compute_adjust_gamma`` function, which operates on images). |
| 72 |
|
| 73 |
- The test function should be marked with the ``@pytest.mark.validation`` decorator. |
| 74 |
|
| 75 |
Following those rules ensures that: |
| 76 |
|
| 77 |
- The tests are easily identified as technical validation tests. |
| 78 |
|
| 79 |
- The tests can be executed separately using the command line interface |
| 80 |
(see :ref:`run_scientific_validation_tests`). |
| 81 |
|
| 82 |
- The tests are automatically discovered for synthetizing the validation status of |
| 83 |
the compute functions (see :ref:`validation_status`). |
| 84 |
|
| 85 |
Executing tests |
| 86 |
--------------- |
| 87 |
|
| 88 |
In DataLab, technical validation tests are disseminated in the test suite of the |
| 89 |
project, but they can also be executed separately using the command line interface. |
| 90 |
|
| 91 |
.. seealso:: |
| 92 |
|
| 93 |
See paragraph :ref:`run_scientific_validation_tests` for more information on |
| 94 |
how to run technical validation tests. |