Debian Codegraph @

doc/intro/tutorials/custom_func.rst

raw
1 .. _tutorial_custom_func:
2
3 :octicon:`book;1em;sd-text-info` Prototyping a custom processing pipeline
4 =========================================================================
5
6 .. meta::
7 :description: Tutorial on how to prototype a custom image processing pipeline using DataLab (macro, plugin, external IDE, Jupyter notebook)
8 :keywords: DataLab, custom function, macro-command, plugin, image processing, prototype, Spyder, Jupyter
9
10 For your specific application, it is possible that you will need a function that is not
11 available in DataLab by default. In this case, you can define your own function and
12 integrate it in DataLab to conveniently analyze your data. DataLab has been designed
13 to be flexible and extensible, and the goal of this tutorial is to show you how to do
14 that.
15
16 This example shows how to prototype a custom image processing pipeline using DataLab:
17
18 - Define a custom processing function
19 - Create a macro-command to apply the function to an image
20 - Use the same code from an external IDE (e.g. Spyder) or a Jupyter notebook
21 - Create a plugin to integrate the function in the DataLab GUI
22
23
24 Define a custom processing function
25 -----------------------------------
26
27 To illustrate the extensibility of DataLab, we will use a simple image processing
28 function that is not available in the standard DataLab distribution, and that
29 represents a typical use case for prototyping a custom processing pipeline.
30
31 The function that we will work on is a denoising filter that combines the ideas of
32 averaging and edge detection. This filter will average the pixel values in the
33 neighborhood, but with a twist: it will give less weight to pixels that are
34 significantly different from the central pixel, assuming they might be part of an
35 edge or noise. Once you have understood how to implement custom processing in
36 DataLab, you will define your own functions adapted to your specific needs.
37
38 Here is the code of the ``weighted_average_denoise`` function::
39
40 def weighted_average_denoise(data: np.ndarray) -> np.ndarray:
41 """Apply a custom denoising filter to an image.
42
43 This filter averages the pixels in a 5x5 neighborhood, but gives less weight
44 to pixels that significantly differ from the central pixel.
45 """
46
47 def filter_func(values: np.ndarray) -> float:
48 """Filter function"""
49 central_pixel = values[len(values) // 2]
50 differences = np.abs(values - central_pixel)
51 weights = np.exp(-differences / np.mean(differences))
52 return np.average(values, weights=weights)
53
54 return spi.generic_filter(data, filter_func, size=5)
55
56
57 Setup the test environment
58 --------------------------
59
60 To test our processing function, we will use an image generated from a DataLab
61 plugin example (`plugins/examples/datalab_example_imageproc.py`). Before starting,
62 make sure that the plugin is installed in DataLab (see the first steps of the
63 tutorial :ref:`tutorial_blobs` to understand how to do it).
64
65 We then reorganize the DataLab window layout to have a comfortable environment for
66 developing and testing our function: we reorganize the window layout of DataLab to have
67 the "Image Panel" on the left and the "Macro Panel" on the right. To do so, you can
68 simply drag and drop the panels to the desired position. If the
69 "Macro Panel" is not visible, you can enable it from the "View > Panels > Macro Panel".
70
71 .. figure:: ../../images/tutorials/custom_func/01.png
72
73 The window layout of DataLab reorganized to have the "Image Panel" on
74 the left and the "Macro Panel" on the right.
75
76 We can now generate a test image using the plugin that we installed before. To do so,
77 we click on the "Plugins > Extract blobs (example) > Generate test image" menu. The
78 function we are developing is quite slow, so we choose a limited size for the image
79 (for example 512x512 pixels).
80
81 .. figure:: ../../images/tutorials/custom_func/02.png
82
83 We generate a new image using the
84 "Plugins > Extract blobs (example) > Generate test image" menu.
85
86 .. figure:: ../../images/tutorials/custom_func/03.png
87
88 We select the size for the image and click on "OK".
89
90 The plugin generates a test image and adds it to the "Image Panel".
91
92 .. figure:: ../../images/tutorials/custom_func/04.png
93
94 The generated image in the "Image Panel".
95
96
97 Create a macro-command
98 ----------------------
99
100 A macro is a code that can be executed directly inside DataLab to automate tasks.
101 Macros are written in Python, and use the DataLab API to interact with DataLab.
102
103 In addition, macros have the following important features:
104
105 - Macros are part of DataLab's **workspace**, which means that they are saved
106 and restored when exporting and importing to/from an HDF5 file.
107
108 - Macros are executed in a separate process, so we need to import the necessary
109 modules and initialize the proxy to DataLab. The proxy is a special object that
110 allows us to communicate with DataLab.
111
112 - As a consequence, **when defining a plugin or when controlling DataLab from an
113 external IDE, we can use exactly the same code as in the macro-command**. This
114 is a very important point, because it means that we can prototype our processing
115 pipeline in DataLab, and then use the same code in a plugin or in an external IDE
116 to develop it further.
117
118 .. note::
119
120 The macro-command is executed in DataLab's Python environment, so we can use
121 the modules that are available in DataLab. However, we can also use our own
122 modules, as long as they are installed in DataLab's Python environment or in
123 a Python distribution that is compatible with DataLab's Python environment.
124 A list of the modules that are available in your DataLab's Python environment
125 is available in the "? > Installation and configuration >
126 Installation configuration" dialog.
127
128 If your custom modules are not installed in DataLab's Python environment, and
129 if they are compatible with DataLab's Python version, you can prepend the
130 ``sys.path`` with the path to the Python distribution that contains your
131 modules::
132
133 import sys
134 sys.path.insert(0, "/path/to/my/python/distribution")
135
136 This will allow you to import your modules in the macro-command and mix them
137 with the modules that are available in DataLab.
138
139 .. warning::
140
141 If you use this method, make sure that your modules are compatible with
142 DataLab's Python version. Otherwise, you will get errors when importing
143 them.
144
145 To edit macros, we use the "Macro Panel" that is available in DataLab. This panel is
146 separated in two parts: the upper part is the macro editor, where we can write and edit
147 the macro code; the lower part is the console, where we can see the output of the macro.
148 If the macro editor is too small to show all the buttons of the macro toolbar, some of
149 them are hidden and can be accessed by clicking on the unfold button. Alternatively, you
150 can resize the macro editor by dragging the splitter between the editor and the console.
151
152 Let's get back to our custom function. We can create a new macro-command that will
153 apply the function to the current image. To do so, we open the "Macro Panel" and
154 click on the "New macro" |libre-gui-add| button.
155
156 .. |libre-gui-add| image:: ../../../datalab/data/icons/libre-gui-add.svg
157 :width: 24px
158 :height: 24px
159 :class: dark-light no-scaled-link
160
161 DataLab creates a new macro-command which is not empty: it contains a sample code
162 that shows how to create a new image and add it to the "Image Panel".
163
164 In this code, we can notice the presence of the ``RemoteProxy`` class that is used to
165 communicate with DataLab. This class is part of the DataLab API, and is used to access
166 DataLab objects and functions from a macro-command, a plugin, or an external IDE.
167 You can find the documentation of the DataLab API in the :ref:`api` section.
168
169 We can remove this code and replace it with our own code::
170
171 # Import the necessary modules
172 import numpy as np
173 import scipy.ndimage as spi
174 from datalab.control.proxy import RemoteProxy
175
176 # Define our custom processing function
177 def weighted_average_denoise(values: np.ndarray) -> float:
178 """Apply a custom denoising filter to an image.
179
180 This filter averages the pixels in a 5x5 neighborhood, but gives less weight
181 to pixels that significantly differ from the central pixel.
182 """
183 central_pixel = values[len(values) // 2]
184 differences = np.abs(values - central_pixel)
185 weights = np.exp(-differences / np.mean(differences))
186 return np.average(values, weights=weights)
187
188 # Initialize the proxy to DataLab
189 proxy = RemoteProxy()
190
191 # Switch to the "Image Panel" and get the current image
192 proxy.set_current_panel("image")
193 image = proxy.get_object()
194 if image is None:
195 # We raise an explicit error if there is no image to process
196 raise RuntimeError("No image to process!")
197
198 # Get a copy of the image data, and apply the function to it
199 data = np.array(image.data, copy=True)
200 data = spi.generic_filter(data, weighted_average_denoise, size=5)
201
202 # Add new image to the panel
203 proxy.add_image("My custom filtered data", data)
204
205
206 Now, let's execute the macro-command by clicking on the "Run macro"
207 |libre-camera-flash-on| button:
208
209 - The macro-command is executed in a separate process, so we can continue to
210 work in DataLab while the macro-command is running. And, if the macro-command
211 takes too long to execute, we can stop it by clicking on the "Stop macro"
212 |libre-camera-flash-off| button.
213
214 - During the execution of the macro-command, we can see the progress in the
215 "Macro Panel" window: the process standard output is displayed in the
216 "Console" below the macro editor. We can see the following messages:
217
218 - ``---[...]---[# ==> Running 'Untitled 01' macro...]``: the macro-command starts
219 - ``Connecting to DataLab XML-RPC server...OK [...]``: the proxy is connected to DataLab
220 - ``---[...]---[# <== 'Untitled 01' macro has finished]``: the macro-command ends
221
222 .. |libre-camera-flash-on| image:: ../../../datalab/data/icons/libre-camera-flash-on.svg
223 :width: 24px
224 :height: 24px
225 :class: dark-light no-scaled-link
226
227 .. |libre-camera-flash-off| image:: ../../../datalab/data/icons/libre-camera-flash-off.svg
228 :width: 24px
229 :height: 24px
230 :class: dark-light no-scaled-link
231
232 .. figure:: ../../images/tutorials/custom_func/05.png
233
234 When the macro-command has finished, we can see the new image in the "Image Panel".
235 Our filter has been applied to the image, and we can see that the noise has been
236 reduced.
237
238
239 Prototyping with an external IDE
240 --------------------------------
241
242 Now that we have a working prototype of our processing pipeline, we can use the same
243 code in an external IDE to develop it further.
244
245 For example, we can use the Spyder IDE to debug our code. To do so, we need to
246 install Spyder but not necessarily in DataLab's Python environment (in the case
247 of the stand-alone version of DataLab, it wouldn't be possible anyway).
248
249 The only requirement is to install a DataLab client in Spyder's Python environment:
250
251 - If you use the stand-alone version of DataLab or if you want or need to keep
252 DataLab and Spyder in separate Python environments, you can install the
253 `Sigima <https://pypi.org/project/sigima/>`_ (``sigima``) package that provides a
254 simple remote client for DataLab. To install it, just run:
255
256 pip install sigima
257
258 Or you may also install the `DataLab Python package <https://pypi.org/project/datalab-platform/>`_
259 (``datalab``) which includes the client (but also other modules, so we don't recommend
260 this method if you don't need all DataLab's features in this Python environment)::
261
262 pip install datalab-platform
263
264 - If you use the DataLab Python package, you may run Spyder in the same Python
265 environment as DataLab, so you don't need to install the client: it is already
266 available in the main DataLab package (the ``datalab`` package).
267
268 Once the client is installed, we can start Spyder and create a new Python script:
269
270 .. literalinclude:: custom_func.py
271 :language: python
272 :linenos:
273
274 We go back to DataLab and select the first image in the "Image Panel".
275
276
277 .. figure:: ../../images/tutorials/custom_func/06.png
278
279 The first image in the "Image Panel" selected.
280
281 Then, we execute the script in Spyder (or your preferred editor), step-by-step (using
282 the defined cells), and we can see the result in DataLab.
283
284 .. figure:: ../../images/tutorials/custom_func/07.png
285
286 The Spyder editor environment with the custom processing script.
287
288 We can see in DataLab that a new image has been added to the "Image Panel". This
289 image is the result of the execution of the script in Spyder.
290
291 Here we have used the script without any modification, but we could have modified it
292 to test new ideas, and then use the modified script in DataLab.
293
294 There is another useful application of having an external script interfaced with
295 DataLab: you can imagine letting your external script acquire the data and add
296 it to DataLab automatically, for example in an acquisition loop.
297 In this way you can improve your measurement workflow with the possibility to visualize
298 and analyze data directly after their acquisition.
299
300
301 Prototyping with a Jupyter notebook
302 -----------------------------------
303
304 We can also use a Jupyter notebook to prototype our processing pipeline. To do so,
305 we need to install Jupyter but not necessarily in DataLab's Python environment
306 (in the case of the stand-alone version of DataLab, it wouldn't be possible anyway).
307
308 The procedure is the same as for Spyder or any other IDE like Visual Studio Code, for example.
309
310 .. figure:: ../../images/tutorials/custom_func/nb.png
311
312 A code interfaced with datalab written in a Jupyter notebook.
313
314
315 Creating a plugin
316 -----------------
317
318 Now that we have a working prototype of our processing pipeline, we can choose to create a
319 plugin to integrate it in DataLab's GUI. To do so, we need to create a new Python
320 module that will contain the plugin code. We can use the same code as in the
321 macro-command, but we need to make some changes.
322
323 .. seealso::
324
325 The plugin system is described in the :ref:`about_plugins` section.
326
327 Apart from integrating the feature to DataLab's GUI which is more convenient for
328 the user, the advantage of creating a plugin is that we can take benefit of the
329 DataLab infrastructure if we encapsulate our processing function in a certain
330 way (see below):
331
332 - Our function will be executed in a separate process, so we can interrupt it if it
333 takes too long to execute.
334
335 - Warnings and errors will be handled by DataLab, so we don't need to handle them
336 ourselves.
337
338 The most significant change we need to make to our code is that we need to define a
339 function that will be operating on DataLab's native image objects
340 (:class:`sigima.objects.ImageObj`), instead of operating on NumPy arrays.
341 So we need to find a way to call our custom function
342 ``weighted_average_denoise`` with a :class:`sigima.objects.ImageObj` as input
343 and output. To avoid writing a lot of boilerplate code, we can use the function wrapper provided by DataLab: :class:`sigima.proc.image.Wrap1to1Func`.
344
345 Besides we need to define a class that describes our plugin, which must inherit
346 from :class:`datalab.plugins.PluginBase` and name the Python script that contains the
347 plugin code with a name that starts with ``datalab_`` (e.g., ``datalab_custom_func.py``), so
348 that DataLab can discover it at startup.
349
350 Moreover, inside the plugin code, we want to add an entry in the "Plugins" menu, so
351 that the user can access our plugin from the GUI.
352
353 Here is the plugin code:
354
355 .. literalinclude:: ../../../plugins/examples/datalab_custom_func.py
356 :language: python
357 :linenos:
358
359 To test it, we have to add the plugin script to one of the plugin directories that
360 are discovered by DataLab at startup: you can find it under "Files > Settings"
361 (see the :ref:`about_plugins` section for more details, or the :ref:`tutorial_blobs`
362 for an example). We then restart DataLab and we can see that the plugin has been loaded.
363
364 .. figure:: ../../images/tutorials/custom_func/10.png
365
366 At the restart we can see that the plugin has been loaded.
367
368 .. figure:: ../../images/tutorials/custom_func/11.png
369
370 We generate again our test image using (see the first steps of the tutorial),
371 and we process it using the plugin: "Plugins > My custom filters > Weighted average denoise".