| 1 |
.. _tutorial_custom_func: |
| 2 |
|
| 3 |
:octicon:`book;1em;sd-text-info` Prototyping a custom processing pipeline |
| 4 |
========================================================================= |
| 5 |
|
| 6 |
.. meta:: |
| 7 |
:description: Tutorial on how to prototype a custom image processing pipeline using DataLab (macro, plugin, external IDE, Jupyter notebook) |
| 8 |
:keywords: DataLab, custom function, macro-command, plugin, image processing, prototype, Spyder, Jupyter |
| 9 |
|
| 10 |
For your specific application, it is possible that you will need a function that is not |
| 11 |
available in DataLab by default. In this case, you can define your own function and |
| 12 |
integrate it in DataLab to conveniently analyze your data. DataLab has been designed |
| 13 |
to be flexible and extensible, and the goal of this tutorial is to show you how to do |
| 14 |
that. |
| 15 |
|
| 16 |
This example shows how to prototype a custom image processing pipeline using DataLab: |
| 17 |
|
| 18 |
- Define a custom processing function |
| 19 |
- Create a macro-command to apply the function to an image |
| 20 |
- Use the same code from an external IDE (e.g. Spyder) or a Jupyter notebook |
| 21 |
- Create a plugin to integrate the function in the DataLab GUI |
| 22 |
|
| 23 |
|
| 24 |
Define a custom processing function |
| 25 |
----------------------------------- |
| 26 |
|
| 27 |
To illustrate the extensibility of DataLab, we will use a simple image processing |
| 28 |
function that is not available in the standard DataLab distribution, and that |
| 29 |
represents a typical use case for prototyping a custom processing pipeline. |
| 30 |
|
| 31 |
The function that we will work on is a denoising filter that combines the ideas of |
| 32 |
averaging and edge detection. This filter will average the pixel values in the |
| 33 |
neighborhood, but with a twist: it will give less weight to pixels that are |
| 34 |
significantly different from the central pixel, assuming they might be part of an |
| 35 |
edge or noise. Once you have understood how to implement custom processing in |
| 36 |
DataLab, you will define your own functions adapted to your specific needs. |
| 37 |
|
| 38 |
Here is the code of the ``weighted_average_denoise`` function:: |
| 39 |
|
| 40 |
def weighted_average_denoise(data: np.ndarray) -> np.ndarray: |
| 41 |
"""Apply a custom denoising filter to an image. |
| 42 |
|
| 43 |
This filter averages the pixels in a 5x5 neighborhood, but gives less weight |
| 44 |
to pixels that significantly differ from the central pixel. |
| 45 |
""" |
| 46 |
|
| 47 |
def filter_func(values: np.ndarray) -> float: |
| 48 |
"""Filter function""" |
| 49 |
central_pixel = values[len(values) // 2] |
| 50 |
differences = np.abs(values - central_pixel) |
| 51 |
weights = np.exp(-differences / np.mean(differences)) |
| 52 |
return np.average(values, weights=weights) |
| 53 |
|
| 54 |
return spi.generic_filter(data, filter_func, size=5) |
| 55 |
|
| 56 |
|
| 57 |
Setup the test environment |
| 58 |
-------------------------- |
| 59 |
|
| 60 |
To test our processing function, we will use an image generated from a DataLab |
| 61 |
plugin example (`plugins/examples/datalab_example_imageproc.py`). Before starting, |
| 62 |
make sure that the plugin is installed in DataLab (see the first steps of the |
| 63 |
tutorial :ref:`tutorial_blobs` to understand how to do it). |
| 64 |
|
| 65 |
We then reorganize the DataLab window layout to have a comfortable environment for |
| 66 |
developing and testing our function: we reorganize the window layout of DataLab to have |
| 67 |
the "Image Panel" on the left and the "Macro Panel" on the right. To do so, you can |
| 68 |
simply drag and drop the panels to the desired position. If the |
| 69 |
"Macro Panel" is not visible, you can enable it from the "View > Panels > Macro Panel". |
| 70 |
|
| 71 |
.. figure:: ../../images/tutorials/custom_func/01.png |
| 72 |
|
| 73 |
The window layout of DataLab reorganized to have the "Image Panel" on |
| 74 |
the left and the "Macro Panel" on the right. |
| 75 |
|
| 76 |
We can now generate a test image using the plugin that we installed before. To do so, |
| 77 |
we click on the "Plugins > Extract blobs (example) > Generate test image" menu. The |
| 78 |
function we are developing is quite slow, so we choose a limited size for the image |
| 79 |
(for example 512x512 pixels). |
| 80 |
|
| 81 |
.. figure:: ../../images/tutorials/custom_func/02.png |
| 82 |
|
| 83 |
We generate a new image using the |
| 84 |
"Plugins > Extract blobs (example) > Generate test image" menu. |
| 85 |
|
| 86 |
.. figure:: ../../images/tutorials/custom_func/03.png |
| 87 |
|
| 88 |
We select the size for the image and click on "OK". |
| 89 |
|
| 90 |
The plugin generates a test image and adds it to the "Image Panel". |
| 91 |
|
| 92 |
.. figure:: ../../images/tutorials/custom_func/04.png |
| 93 |
|
| 94 |
The generated image in the "Image Panel". |
| 95 |
|
| 96 |
|
| 97 |
Create a macro-command |
| 98 |
---------------------- |
| 99 |
|
| 100 |
A macro is a code that can be executed directly inside DataLab to automate tasks. |
| 101 |
Macros are written in Python, and use the DataLab API to interact with DataLab. |
| 102 |
|
| 103 |
In addition, macros have the following important features: |
| 104 |
|
| 105 |
- Macros are part of DataLab's **workspace**, which means that they are saved |
| 106 |
and restored when exporting and importing to/from an HDF5 file. |
| 107 |
|
| 108 |
- Macros are executed in a separate process, so we need to import the necessary |
| 109 |
modules and initialize the proxy to DataLab. The proxy is a special object that |
| 110 |
allows us to communicate with DataLab. |
| 111 |
|
| 112 |
- As a consequence, **when defining a plugin or when controlling DataLab from an |
| 113 |
external IDE, we can use exactly the same code as in the macro-command**. This |
| 114 |
is a very important point, because it means that we can prototype our processing |
| 115 |
pipeline in DataLab, and then use the same code in a plugin or in an external IDE |
| 116 |
to develop it further. |
| 117 |
|
| 118 |
.. note:: |
| 119 |
|
| 120 |
The macro-command is executed in DataLab's Python environment, so we can use |
| 121 |
the modules that are available in DataLab. However, we can also use our own |
| 122 |
modules, as long as they are installed in DataLab's Python environment or in |
| 123 |
a Python distribution that is compatible with DataLab's Python environment. |
| 124 |
A list of the modules that are available in your DataLab's Python environment |
| 125 |
is available in the "? > Installation and configuration > |
| 126 |
Installation configuration" dialog. |
| 127 |
|
| 128 |
If your custom modules are not installed in DataLab's Python environment, and |
| 129 |
if they are compatible with DataLab's Python version, you can prepend the |
| 130 |
``sys.path`` with the path to the Python distribution that contains your |
| 131 |
modules:: |
| 132 |
|
| 133 |
import sys |
| 134 |
sys.path.insert(0, "/path/to/my/python/distribution") |
| 135 |
|
| 136 |
This will allow you to import your modules in the macro-command and mix them |
| 137 |
with the modules that are available in DataLab. |
| 138 |
|
| 139 |
.. warning:: |
| 140 |
|
| 141 |
If you use this method, make sure that your modules are compatible with |
| 142 |
DataLab's Python version. Otherwise, you will get errors when importing |
| 143 |
them. |
| 144 |
|
| 145 |
To edit macros, we use the "Macro Panel" that is available in DataLab. This panel is |
| 146 |
separated in two parts: the upper part is the macro editor, where we can write and edit |
| 147 |
the macro code; the lower part is the console, where we can see the output of the macro. |
| 148 |
If the macro editor is too small to show all the buttons of the macro toolbar, some of |
| 149 |
them are hidden and can be accessed by clicking on the unfold button. Alternatively, you |
| 150 |
can resize the macro editor by dragging the splitter between the editor and the console. |
| 151 |
|
| 152 |
Let's get back to our custom function. We can create a new macro-command that will |
| 153 |
apply the function to the current image. To do so, we open the "Macro Panel" and |
| 154 |
click on the "New macro" |libre-gui-add| button. |
| 155 |
|
| 156 |
.. |libre-gui-add| image:: ../../../datalab/data/icons/libre-gui-add.svg |
| 157 |
:width: 24px |
| 158 |
:height: 24px |
| 159 |
:class: dark-light no-scaled-link |
| 160 |
|
| 161 |
DataLab creates a new macro-command which is not empty: it contains a sample code |
| 162 |
that shows how to create a new image and add it to the "Image Panel". |
| 163 |
|
| 164 |
In this code, we can notice the presence of the ``RemoteProxy`` class that is used to |
| 165 |
communicate with DataLab. This class is part of the DataLab API, and is used to access |
| 166 |
DataLab objects and functions from a macro-command, a plugin, or an external IDE. |
| 167 |
You can find the documentation of the DataLab API in the :ref:`api` section. |
| 168 |
|
| 169 |
We can remove this code and replace it with our own code:: |
| 170 |
|
| 171 |
# Import the necessary modules |
| 172 |
import numpy as np |
| 173 |
import scipy.ndimage as spi |
| 174 |
from datalab.control.proxy import RemoteProxy |
| 175 |
|
| 176 |
# Define our custom processing function |
| 177 |
def weighted_average_denoise(values: np.ndarray) -> float: |
| 178 |
"""Apply a custom denoising filter to an image. |
| 179 |
|
| 180 |
This filter averages the pixels in a 5x5 neighborhood, but gives less weight |
| 181 |
to pixels that significantly differ from the central pixel. |
| 182 |
""" |
| 183 |
central_pixel = values[len(values) // 2] |
| 184 |
differences = np.abs(values - central_pixel) |
| 185 |
weights = np.exp(-differences / np.mean(differences)) |
| 186 |
return np.average(values, weights=weights) |
| 187 |
|
| 188 |
# Initialize the proxy to DataLab |
| 189 |
proxy = RemoteProxy() |
| 190 |
|
| 191 |
# Switch to the "Image Panel" and get the current image |
| 192 |
proxy.set_current_panel("image") |
| 193 |
image = proxy.get_object() |
| 194 |
if image is None: |
| 195 |
# We raise an explicit error if there is no image to process |
| 196 |
raise RuntimeError("No image to process!") |
| 197 |
|
| 198 |
# Get a copy of the image data, and apply the function to it |
| 199 |
data = np.array(image.data, copy=True) |
| 200 |
data = spi.generic_filter(data, weighted_average_denoise, size=5) |
| 201 |
|
| 202 |
# Add new image to the panel |
| 203 |
proxy.add_image("My custom filtered data", data) |
| 204 |
|
| 205 |
|
| 206 |
Now, let's execute the macro-command by clicking on the "Run macro" |
| 207 |
|libre-camera-flash-on| button: |
| 208 |
|
| 209 |
- The macro-command is executed in a separate process, so we can continue to |
| 210 |
work in DataLab while the macro-command is running. And, if the macro-command |
| 211 |
takes too long to execute, we can stop it by clicking on the "Stop macro" |
| 212 |
|libre-camera-flash-off| button. |
| 213 |
|
| 214 |
- During the execution of the macro-command, we can see the progress in the |
| 215 |
"Macro Panel" window: the process standard output is displayed in the |
| 216 |
"Console" below the macro editor. We can see the following messages: |
| 217 |
|
| 218 |
- ``---[...]---[# ==> Running 'Untitled 01' macro...]``: the macro-command starts |
| 219 |
- ``Connecting to DataLab XML-RPC server...OK [...]``: the proxy is connected to DataLab |
| 220 |
- ``---[...]---[# <== 'Untitled 01' macro has finished]``: the macro-command ends |
| 221 |
|
| 222 |
.. |libre-camera-flash-on| image:: ../../../datalab/data/icons/libre-camera-flash-on.svg |
| 223 |
:width: 24px |
| 224 |
:height: 24px |
| 225 |
:class: dark-light no-scaled-link |
| 226 |
|
| 227 |
.. |libre-camera-flash-off| image:: ../../../datalab/data/icons/libre-camera-flash-off.svg |
| 228 |
:width: 24px |
| 229 |
:height: 24px |
| 230 |
:class: dark-light no-scaled-link |
| 231 |
|
| 232 |
.. figure:: ../../images/tutorials/custom_func/05.png |
| 233 |
|
| 234 |
When the macro-command has finished, we can see the new image in the "Image Panel". |
| 235 |
Our filter has been applied to the image, and we can see that the noise has been |
| 236 |
reduced. |
| 237 |
|
| 238 |
|
| 239 |
Prototyping with an external IDE |
| 240 |
-------------------------------- |
| 241 |
|
| 242 |
Now that we have a working prototype of our processing pipeline, we can use the same |
| 243 |
code in an external IDE to develop it further. |
| 244 |
|
| 245 |
For example, we can use the Spyder IDE to debug our code. To do so, we need to |
| 246 |
install Spyder but not necessarily in DataLab's Python environment (in the case |
| 247 |
of the stand-alone version of DataLab, it wouldn't be possible anyway). |
| 248 |
|
| 249 |
The only requirement is to install a DataLab client in Spyder's Python environment: |
| 250 |
|
| 251 |
- If you use the stand-alone version of DataLab or if you want or need to keep |
| 252 |
DataLab and Spyder in separate Python environments, you can install the |
| 253 |
`Sigima <https://pypi.org/project/sigima/>`_ (``sigima``) package that provides a |
| 254 |
simple remote client for DataLab. To install it, just run: |
| 255 |
|
| 256 |
pip install sigima |
| 257 |
|
| 258 |
Or you may also install the `DataLab Python package <https://pypi.org/project/datalab-platform/>`_ |
| 259 |
(``datalab``) which includes the client (but also other modules, so we don't recommend |
| 260 |
this method if you don't need all DataLab's features in this Python environment):: |
| 261 |
|
| 262 |
pip install datalab-platform |
| 263 |
|
| 264 |
- If you use the DataLab Python package, you may run Spyder in the same Python |
| 265 |
environment as DataLab, so you don't need to install the client: it is already |
| 266 |
available in the main DataLab package (the ``datalab`` package). |
| 267 |
|
| 268 |
Once the client is installed, we can start Spyder and create a new Python script: |
| 269 |
|
| 270 |
.. literalinclude:: custom_func.py |
| 271 |
:language: python |
| 272 |
:linenos: |
| 273 |
|
| 274 |
We go back to DataLab and select the first image in the "Image Panel". |
| 275 |
|
| 276 |
|
| 277 |
.. figure:: ../../images/tutorials/custom_func/06.png |
| 278 |
|
| 279 |
The first image in the "Image Panel" selected. |
| 280 |
|
| 281 |
Then, we execute the script in Spyder (or your preferred editor), step-by-step (using |
| 282 |
the defined cells), and we can see the result in DataLab. |
| 283 |
|
| 284 |
.. figure:: ../../images/tutorials/custom_func/07.png |
| 285 |
|
| 286 |
The Spyder editor environment with the custom processing script. |
| 287 |
|
| 288 |
We can see in DataLab that a new image has been added to the "Image Panel". This |
| 289 |
image is the result of the execution of the script in Spyder. |
| 290 |
|
| 291 |
Here we have used the script without any modification, but we could have modified it |
| 292 |
to test new ideas, and then use the modified script in DataLab. |
| 293 |
|
| 294 |
There is another useful application of having an external script interfaced with |
| 295 |
DataLab: you can imagine letting your external script acquire the data and add |
| 296 |
it to DataLab automatically, for example in an acquisition loop. |
| 297 |
In this way you can improve your measurement workflow with the possibility to visualize |
| 298 |
and analyze data directly after their acquisition. |
| 299 |
|
| 300 |
|
| 301 |
Prototyping with a Jupyter notebook |
| 302 |
----------------------------------- |
| 303 |
|
| 304 |
We can also use a Jupyter notebook to prototype our processing pipeline. To do so, |
| 305 |
we need to install Jupyter but not necessarily in DataLab's Python environment |
| 306 |
(in the case of the stand-alone version of DataLab, it wouldn't be possible anyway). |
| 307 |
|
| 308 |
The procedure is the same as for Spyder or any other IDE like Visual Studio Code, for example. |
| 309 |
|
| 310 |
.. figure:: ../../images/tutorials/custom_func/nb.png |
| 311 |
|
| 312 |
A code interfaced with datalab written in a Jupyter notebook. |
| 313 |
|
| 314 |
|
| 315 |
Creating a plugin |
| 316 |
----------------- |
| 317 |
|
| 318 |
Now that we have a working prototype of our processing pipeline, we can choose to create a |
| 319 |
plugin to integrate it in DataLab's GUI. To do so, we need to create a new Python |
| 320 |
module that will contain the plugin code. We can use the same code as in the |
| 321 |
macro-command, but we need to make some changes. |
| 322 |
|
| 323 |
.. seealso:: |
| 324 |
|
| 325 |
The plugin system is described in the :ref:`about_plugins` section. |
| 326 |
|
| 327 |
Apart from integrating the feature to DataLab's GUI which is more convenient for |
| 328 |
the user, the advantage of creating a plugin is that we can take benefit of the |
| 329 |
DataLab infrastructure if we encapsulate our processing function in a certain |
| 330 |
way (see below): |
| 331 |
|
| 332 |
- Our function will be executed in a separate process, so we can interrupt it if it |
| 333 |
takes too long to execute. |
| 334 |
|
| 335 |
- Warnings and errors will be handled by DataLab, so we don't need to handle them |
| 336 |
ourselves. |
| 337 |
|
| 338 |
The most significant change we need to make to our code is that we need to define a |
| 339 |
function that will be operating on DataLab's native image objects |
| 340 |
(:class:`sigima.objects.ImageObj`), instead of operating on NumPy arrays. |
| 341 |
So we need to find a way to call our custom function |
| 342 |
``weighted_average_denoise`` with a :class:`sigima.objects.ImageObj` as input |
| 343 |
and output. To avoid writing a lot of boilerplate code, we can use the function wrapper provided by DataLab: :class:`sigima.proc.image.Wrap1to1Func`. |
| 344 |
|
| 345 |
Besides we need to define a class that describes our plugin, which must inherit |
| 346 |
from :class:`datalab.plugins.PluginBase` and name the Python script that contains the |
| 347 |
plugin code with a name that starts with ``datalab_`` (e.g., ``datalab_custom_func.py``), so |
| 348 |
that DataLab can discover it at startup. |
| 349 |
|
| 350 |
Moreover, inside the plugin code, we want to add an entry in the "Plugins" menu, so |
| 351 |
that the user can access our plugin from the GUI. |
| 352 |
|
| 353 |
Here is the plugin code: |
| 354 |
|
| 355 |
.. literalinclude:: ../../../plugins/examples/datalab_custom_func.py |
| 356 |
:language: python |
| 357 |
:linenos: |
| 358 |
|
| 359 |
To test it, we have to add the plugin script to one of the plugin directories that |
| 360 |
are discovered by DataLab at startup: you can find it under "Files > Settings" |
| 361 |
(see the :ref:`about_plugins` section for more details, or the :ref:`tutorial_blobs` |
| 362 |
for an example). We then restart DataLab and we can see that the plugin has been loaded. |
| 363 |
|
| 364 |
.. figure:: ../../images/tutorials/custom_func/10.png |
| 365 |
|
| 366 |
At the restart we can see that the plugin has been loaded. |
| 367 |
|
| 368 |
.. figure:: ../../images/tutorials/custom_func/11.png |
| 369 |
|
| 370 |
We generate again our test image using (see the first steps of the tutorial), |
| 371 |
and we process it using the plugin: "Plugins > My custom filters > Weighted average denoise". |