Debian Codegraph @

doc/features/validation/technical.rst

raw
1 .. _scientific_validation:
2
3 Technical Validation
4 =====================
5
6 .. meta::
7 :description: Validation in DataLab, the open-source scientific data analysis and visualization platform
8 :keywords: DataLab, scientific, data, analysis, validation, ground-truth, analytical
9
10 DataLab technical validation is based on two key concepts:
11 **ground-truth data** and **analytical validation**.
12
13 Ground-truth data
14 -----------------
15
16 Ground-truth data is data that is known to be correct. It is used to validate the
17 results of computational analysis.
18
19 In DataLab, ground-truth data may be obtained from a variety of sources, including:
20
21 - Experimental data
22 - Simulated data
23 - Synthetic data
24 - Data from a trusted source
25
26 Analytical validation
27 ---------------------
28
29 Analytical validation is the process of comparing the results of computational analysis
30 to ground-truth data. This is done to ensure that the results are accurate and reliable.
31
32 In DataLab, analytical validation is implemented using a variety of techniques, including:
33
34 - Cross-validation with an analytical model (from a trusted source,
35 e.g. `SciPy <https://www.scipy.org/>`_ or `NumPy <https://numpy.org/>`_)
36 - Statistical analysis
37 - Visual inspection
38 - Expert review
39
40 Scope
41 -----
42
43 The scope of technical validation in DataLab includes all compute functions that
44 operate on DataLab's signal and image objects (i.e. :class:`sigima.objects.SignalObj` and
45 :class:`sigima.objects.ImageObj`).
46
47 This includes functions for (all functions are named ``compute_<function_name>``):
48
49 - Signal processing (:mod:`sigima.proc.signal`)
50 - Image processing (:mod:`sigima.proc.image`)
51
52 Implementation
53 --------------
54
55 The tests are implemented using the `pytest <https://docs.pytest.org/en/latest/>`_
56 framework.
57
58 When writing a new technical validation test, the following rules should be followed
59 regarding the test function:
60
61 - The test function should be named:
62
63 - ``test_signal_<function_name>`` for signal compute functions
64 - ``test_image_<function_name>`` for image compute functions
65
66 .. note::
67
68 The ``signal`` or ``image`` prefix is used to indicate the type of object that the
69 function operates on. It may be omitted if the function operates exclusively on
70 one type of object (e.g. ``test_adjust_gamma`` is the test function for the
71 ``compute_adjust_gamma`` function, which operates on images).
72
73 - The test function should be marked with the ``@pytest.mark.validation`` decorator.
74
75 Following those rules ensures that:
76
77 - The tests are easily identified as technical validation tests.
78
79 - The tests can be executed separately using the command line interface
80 (see :ref:`run_scientific_validation_tests`).
81
82 - The tests are automatically discovered for synthetizing the validation status of
83 the compute functions (see :ref:`validation_status`).
84
85 Executing tests
86 ---------------
87
88 In DataLab, technical validation tests are disseminated in the test suite of the
89 project, but they can also be executed separately using the command line interface.
90
91 .. seealso::
92
93 See paragraph :ref:`run_scientific_validation_tests` for more information on
94 how to run technical validation tests.